Exploit the chunkification cache to detect disk errors about tarsnap HOT 4 OPEN

Use case (from personal experience): I used to keep my email and personal files on a SD card so I could check it at home, at university, etc., without having to carry a laptop around. Unfortunately there was some corruption in the SD card (probably from the crappy $1 sd-to-usb device I bought), and tarsnap faithfully backed up the partially-damaged data. It took me a while to find the last good backup, then manually select which changes to which files were good (I'd worked on them in the past year) and which were bad (due to disk corruption).

I'm looking forward to adding this feature to tarsnap, since it would have avoided some personal pain, and it also plays into "backups for the truly paranoid".

from tarsnap.

Colin's feature description (from Jul 14) is addressed in #94. My "use case" (Aug 8) is irrelevant and somewhat misleading: in my case, the mtime and/or size of the files must have changed in order for tarsnap to back up the (corrupted) files. It's still possible that PR #94 would have given me a warning, but it's not guaranteed even if the code existed and I used a check probability of 1.0.

In order to check for filesize/mtime-changing disk corruption, I'd need a different approach.

from tarsnap.

Was your file changing every time you backed it up? If not, there's a good chance that this feature would have warned you when the file was corrupted and you hadn't (deliberately) modified it yet.

But if a file has been modified deliberately, there's no way tarsnap will ever be able to distinguish between "modified deliberately" and "modified deliberately and also corrupted at the same time".

from tarsnap.

I didn't keep careful notes. Sometime around March 2015 (I think?), I discovered that some of my lilypond files didn't compile because the text was corrupted. Further investigation revealed that some old emails also had random corruption (in email bodies and attachments). I checked the previous archive, and it showed the same corruption. I went a few farther back, and I still found it. I think I jumped back to around August 2013 to get the uncorrupted old emails... but remember that I only back up (manually) once every 1-5 months. I'm pretty certain I still have all those archives, so I could try to pin down more details if you want.

The main reason I added the "it might not have detected it" was to acknowledge that if the filesize or mtime changes, there's no way for the tarsnap client to know whether that was done by the user or disk corruption. To detect those possible flaws, we'd need something else.

from tarsnap.