Remotely backup local backups... about btrbk HOT 7 CLOSED

Yes, btrbk archive works incrementally. For very slow connection, consider setting stream_compress xz.

from btrbk.

I don't know of a simple solution for this either. btrbk does not (yet?) support backuping received subvolumes, mostly because the "parent_uuid" points to something different on received subvolumes (the parent backup) as on snapshots (the r/w parent, "working" subvolume).

I have a little dirty shell script that basically mirrors all received subvolumes (backups) from one disk to another by sending them in the correct order, but as I understand this is not really what you need here, since there would be no cleanup policy on your remote NAS (and you should clean up, leaving around thousands of subvolumes is not very wise).

As a third solution, instead of symlinking the latest backup, you could simply rename it. This way btrbk will not delete it, but should still consider it as valid parent for incremental backups from PC to LAN NAS (at least the last time I checked, will check this again). Something like this:

# 1. find latest
latest=$(ls -d alex.home.* | tail -n1)
# 2. transmit subvolume
btrfs send -p alex.home.*.last-transmitted $latest | ssh [email protected] btrfs receive /backups/alex/alex-pc/
# 3. rotate
# make sure there is always only one alex.home.*.last-transmitted!!!
mv alex.home.*.last-transmitted $(basename alex.home.*.last-transmitted .last-transmitted)
mv $latest $latest.last-transmitted`

What do you think of this?

Regarding the snapshot script: I'm currently (re-)working user-friendly commands in btrbk (just pushed master), one of it being "btrbk resolve latest", which replaces the ugly --raw-output solution I proposed earlier. This should make it much easier to script it (also try the different --format=table|long|raw options).
Note that this is all work-in-progress, and the options as well as the output is subject to change before releasing v0.21.

PS: Note that kernels >=4.1 have a bug when re-sending subvolumes, see this thread:
http://thread.gmane.org/gmane.comp.file-systems.btrfs/48798

from btrbk.

The rename thing seems to be a good idea !
But, I use the last stable kernel... 4.2.3, so my exact situation falls into this regression... (BTY thanks a lot to have noticed this, I'd have spend a lot of time failing to send my volumes without knowing why...)
So I will try this when new kernel get released (hopefully the patch will be include in 4.3)

Until then, I will try a cp --reflink=always into a fresh new subvolume used to send to the remote NAS...

I'll keep you informed when I can try your method !
Thank you !

from btrbk.

I just checked the btrbk behaviour with renamed snapshots (40a384e on master branch):

• renamed snapshots are used as parents for btrfs send
• renamed snapshots are not deleted
• renamed snapshots are not resumed

Until then, I will try a cp --reflink=always into a fresh new subvolume used to send to the remote NAS...

I have this patch applied into my 4.2.3 kernel, and it works as expected. So in case you build your kernel by yourself, I can recommend it.

from btrbk.

Good news for the btrbk behavior. I was indeed wondering if renamed snapshots were resumed...

As for the kernel patch, I'd like to keep things simple... I get the last kernel from debian sid.
I learned to (cross)compile a kernel just a month ago, only the basics: I only know make menuconfig and make... Patching must not be that hard, but I will lack a lot of time in the near future, so I've not really enough time to learn, build and keep a home-made kernel up to date (apt is my friend :) )

Thank you for your time and testings!
As soon as I can try this method, I'll post a feedback

from btrbk.

Backups of backups are covered by the (still experimental, but working) btrbk archive <src> <dst> command since btrbk-0.23.0.

from btrbk.

Does archive work incrementally?
I would like to keep my "main" backups at home, then, at night, mirror them to another computer connected to the internet through 7mbps DSL. Backups must be incremental, or it would take days for them to finish.

from btrbk.