Broken .tar file after unpacking about android-backup-extractor HOT 32 OPEN

new android version backup struct changed, try backup with -nocompress

from android-backup-extractor.

try backup with -nocompress

Thank you, that solved my problem. At least, for the moment.

As ABE should be able to handle compressed backup files of newer formats, I leave this issue open.

from android-backup-extractor.

Any idea what the exact change is? This seems to only happen when -shared is specified.

from android-backup-extractor.

I've found similar problem, is the same problem or is not?

Sony Xperia Z5 Compact fails to adb backup shared storage

from android-backup-extractor.

Does -nocompress help? If so, might be related.

from android-backup-extractor.

I've also got this exact problem. Both star and tar are getting the same error.

My phone is now dead so can't generate a new .ab. Any ideas on how I can get abe to work?

from android-backup-extractor.

My phone worked again for a few hours today. I created a new encrypted (is forced on) backup using -nocompress.

The abe unpack command completed without error. The tarball is still broken.

from android-backup-extractor.

@egalanos what is the error of the tar?

from android-backup-extractor.

% ls -l adb-nocompress-unpacked.tar 
-rw-rw-r--. 1 user user 4440329216 Jul  9 14:33 adb-nocompress-unpacked.tar

tar:

% tar tf adb-nocompress-unpacked.tar
<file list>
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

star:

star tvf adb-nocompress-unpacked.tar -debug
<file list>
star: Tar file too small (amount: 0 bytes).
star: Unexpected EOF on input.
star: Cannot recover from error - exiting.
star: 433625 blocks + 9216 bytes (total of 4440329216 bytes = 4336259.00k).

from android-backup-extractor.

Make a backup without excluding external storage (-sdcard), it seems the error happens during unpacking the SD card data.

from android-backup-extractor.

@egalanos file a bug here:
https://issuetracker.google.com/

Similar issues:
https://issuetracker.google.com/issues/36987260
https://issuetracker.google.com/issues/38290747
https://issuetracker.google.com/issues/37150449
https://issuetracker.google.com/issues/37130390
https://issuetracker.google.com/issues/37115971
https://issuetracker.google.com/issues/37103304
https://forum.xda-developers.com/showpost.php?p=20965332&postcount=25
https://issuetracker.google.com/issues/37096097

Just do a search on android bugs.

from android-backup-extractor.

@clhunsen Probably it's an issue with your phone. Try running adb logcat while backup and watch for letter capital E.

• adb logcat -s BackupManagerService

Open another console and backup

• adb backup -apk -shared -all -system

from android-backup-extractor.

same error here... The unpacked .tar file is damaged.

from android-backup-extractor.

@M0rtale android has unfixed bugs

from android-backup-extractor.

@clhunsen Probably it's an issue with your phone. Try running adb logcat while backup and watch for letter capital E.

• adb logcat -s BackupManagerService

Thanks for that hint, @gsgrippal! I found that the Amazon Prime Video app threw some errors. After resetting it (and removing all stored local data for it), the backup ran flawlessly.

from android-backup-extractor.

@clhunsen glad that you found the culprit. Thanks also for the heads up, as you can see adb backup still need a lot of improvement to be user friendly.

from android-backup-extractor.

Any chance to recover a backup that produces a tar which fails like this (I cannot make a new one)?

tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now

Even just a partial recovery of a particular file (I know the paths) would be very helpful.

from android-backup-extractor.

Getting the files using a binary editor, or try using a scripting language to try the read specifics files could work. Assuming the files you need were backed up correctly.

from android-backup-extractor.

@mjakubicek what is the size of your tar file?

from android-backup-extractor.

Seeing tar: Unexpected EOF in archive I tried adb backup -nocompress … - Now the tar of the backup contains shared stuff too, not just apps. But trying to extract still reports errors.

The first tar error happens after a file that is more than 2GB in size. Maybe that gives a pointer to the problem source? The list of files in the tar from the compressed backup ends with this very same file, up to there, both lists are fully identic. Option -i with tar does not help.

2124660736 2020-02-24 16:04 apps/…/bigFile
tar: Skipping to next header
503800 2020-03-09 21:12 shared/0/DCIM/…

UPDATE - adb shell ls bigFile also reports 212…, and that is still less than 2^31 Note: native android Files app reports the size of bigFile as 2.12GB, while tar shows it as 1.98GB (212… above)

Note2: As the shared files are only a fifth of the full ab backup, the difference in ab file size maybe due to compression alone?

from android-backup-extractor.

@hungerburg don't include shared/* on any adb backup, it's broken and will always be. Use -noshared to make adb backup. You can backup your internal sdcard by copying everything to USB OTG. Using MTP may cause some files to be skipped and not be properly copied to your computer. Alternatively user FTP client on phone and server on your computer. Include all hidden files.

from android-backup-extractor.

@mjakubicek what is the size of your tar file?

5.2 GB (yes, with -shared, I didn't know that would be so broken)

from android-backup-extractor.

@gsgrippal I made a comparison: find . on the MTP mount produces the exact same listing as a adb shell find /sdcard0. The tar extracted from ab file indeed is missing some files from that: namely everything below /Android and curiously also /Movies. The rest though looks fine.

Looking further, some of the stuff below MTP/Android is in TAR/apps. So likely it is only the /Movies I should take extra care (I don't :) Additionally, a single command for restore is nice, and modification dates on pictures are nice too (MTP kills those).

Remains the question: what is "tar tf" complaining about?

from android-backup-extractor.

@gsgrippal I made a comparison: find . on the MTP mount produces the exact same listing as a adb shell find /sdcard0. The tar extracted from ab file indeed is missing some files from that: namely everything below /Android and curiously also /Movies. The rest though looks fine.

Looking further, some of the stuff below MTP/Android is in TAR/apps. So likely it is only the /Movies I should take extra care (I don't :) Additionally, a single command for restore is nice, and modification dates on pictures are nice too (MTP kills those).

Remains the question: what is "tar tf" complaining about?

adb unpack what shows? As nelenkov said before, your adb backup was truncated at creation time, then tar is corrupt and truncated as well.

Each time you make some adb backup, test the tar with tar tf. If fails, repeat until works, or eventually use another method.

Ii's not that difficult to understand that android has bugs, in this case is unrelated to android-backup-extractor.

from android-backup-extractor.

Thank you for the hard work and open source tools. I have a rooted pixel3a on Android 10 and the tar from 'abe-all.jar' (20181012025725-d750899). tar gives no output. winrar say tar is invalid. I haven't figured out good error/trace info. Will try some of the techniques above for watching logs during the backup and see if anything comes out there. I am only backing up one app that has allow-backups true. ( ANDROID BACKUP 5 0 none ). Maybe it's just empty?

from android-backup-extractor.

Apps can forbid backup, so in that case behavior is somewhat undefined. How big are the .ab and .tar files? Try looking at contents with editor/cat to get some idea.

from android-backup-extractor.

@gsgrippal […]

Each time you make some adb backup, test the tar with tar tf. If fails, repeat until works, or eventually use another method.

An adb backup done yesterday: The tar received from the extractor tested without error. If I fully understand, the tar is created by adb or even on device, and abe only decrypts the stream from the .ab file.

This is not the forum for android bugs, still of note probably, just for reference:

The tar error was reported immediately following a (very close to) 2GB size file below /apps/, see above "tar: Skipping to next header"; that file meanwhile shrunk considerably; no more error but some more files in /apps get listed.

from android-backup-extractor.

I got the same error. Using

bsdtar

instead gave decompression without error.

from android-backup-extractor.

I got the same error. Using

bsdtar

instead gave decompression without error.

That doesn't even mean anything. Were the files there? It could just exit without message even if file is damaged.

from android-backup-extractor.

Is there any way to restore such broken backup? I had no idea it could be broken and this is the only backup I have before unlocking bootloader and installing lineageos. Restoring it as is doesn't really restore anything, despite taking significant amount of time, and trying to unpack it and pack it again, it "completes" almost immediately and does nothing.

from android-backup-extractor.

@Barteks2x From my limited perspective, it is quite certain, that you cannot use the backup file for restoring. But you can use your favourite archiver program to extract at least some of the data from the tar file that you get from decrypting the backup file. In my case, as I see it, the tar file (and most certainly the backup file too) was broken, because the on device android tool does not handle files, that are larger than 2GB, so everything that got added to the backup before that file would not be lost, just not available for convenient restore.

from android-backup-extractor.

@Barteks2x could you check the alternative tool android-backup-processor? It claims to have improvements about this... but to be honest it's been months since I don't use either tool. Really android backups are no longer needed or useful, it's just a joke.

from android-backup-extractor.