Comments (2)
yoshtec
commented on July 28, 2024
I updated the script, with better handling of the decoding of terminal output. Please check if it works for you
from snapbtrex.
unix1984
commented on July 28, 2024
Hi yoshtec!
TOP!
Thanks, now it's running error-free.
[08:57:39][root@vmd58232:~]$ /usr/local/sbin/snapbtrex.py --snap /mnt/sda3/containers/alpine-znc --path /mnt/sda3/autosnap/alpine-znc/ --target-backups 5 --verbose --remote-host 192.168.10.8 --remote-dir /mnt/vps-lxd-snapshots/alpine-znc/ --remote-keep 15
Local > snapshotting path=/mnt/sda3/containers/alpine-znc to newdir=/mnt/sda3/autosnap/alpine-znc/20210205-075748
EXEC >-> sudo btrfs subvolume snapshot -r /mnt/sda3/containers/alpine-znc /mnt/sda3/autosnap/alpine-znc/20210205-075748
OUTPUT > Create a readonly snapshot of '/mnt/sda3/containers/alpine-znc' in '/mnt/sda3/autosnap/alpine-znc/20210205-075748'
Local > sync filesystem /mnt/sda3/autosnap/alpine-znc/
EXEC >-> sudo btrfs filesystem sync /mnt/sda3/autosnap/alpine-znc/
Local > done sync filesystem /mnt/sda3/autosnap/alpine-znc/
Local > done snapshotting
Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
Remote > send single snapshot from 20210205-075748 to host 192.168.10.8 path=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> sudo btrfs send -v /mnt/sda3/autosnap/alpine-znc/20210205-075748 | pv -brtfL 0 | ssh -p 22 192.168.10.8 'sudo btrfs receive /mnt/vps-lxd-snapshots/alpine-znc/ '
STDERR > At subvol /mnt/sda3/autosnap/alpine-znc/20210205-075748
At subvol 20210205-075748
BTRFS_IOC_SEND returned 0s]
joining genl thread
64.4MiB 0:00:16 [3.89MiB/s]
Remote > last possible parent = 20210205-075748
Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
OUTPUT > 20210205-075748
Remote > No remote directories to clean, currently 1 remote backups, should keep 15
Local > using default value for --keep-backups: 10
Local > Parameters for cleandir: keep_backups=10, target_freespace=None, target_backups=5, max_age=None, keep_latest=False
Local > current amount of backups: 1 have to keep a minimum of 10, stopping further deletionthen I ran another 15x (--keep-backups: 5) and target 15:
[09:08:29][root@vmd58232:~]$ ls -1 /mnt/sda3/autosnap/alpine-znc/
20210205-075748
20210205-075959
20210205-080009
20210205-080016
20210205-080022
20210205-080747
20210205-080752
20210205-080804
20210205-080809
20210205-080827
[09:08:45][root@vmd58232:~]$ nano /usr/local/sbin/snapbtrex.py
[09:09:35][root@vmd58232:~]$ /usr/local/sbin/snapbtrex.py --snap /mnt/sda3/containers/alpine-znc --path /mnt/sda3/autosnap/alpine-znc/ --target-backups 5 --verbose --remote-host 192.168.10.8 --remote-dir /mnt/vps-lxd-snapshots/alpine-znc/ --remote-keep 15
Local > snapshotting path=/mnt/sda3/containers/alpine-znc to newdir=/mnt/sda3/autosnap/alpine-znc/20210205-080937
EXEC >-> sudo btrfs subvolume snapshot -r /mnt/sda3/containers/alpine-znc /mnt/sda3/autosnap/alpine-znc/20210205-080937
OUTPUT > Create a readonly snapshot of '/mnt/sda3/containers/alpine-znc' in '/mnt/sda3/autosnap/alpine-znc/20210205-080937'
Local > sync filesystem /mnt/sda3/autosnap/alpine-znc/
EXEC >-> sudo btrfs filesystem sync /mnt/sda3/autosnap/alpine-znc/
Local > done sync filesystem /mnt/sda3/autosnap/alpine-znc/
Local > done snapshotting
Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
OUTPUT > 20210205-075748
20210205-075959
20210205-080009
20210205-080012
20210205-080016
20210205-080019
20210205-080022
20210205-080747
20210205-080749
20210205-080752
20210205-080756
20210205-080804
20210205-080807
20210205-080809
20210205-080827
Remote > last possible parent = 20210205-080827
Remote > transfer: parent=20210205-080827 snap=20210205-080937
Remote > send snapshot from 20210205-080937 with parent 20210205-080827 to host 192.168.10.8 path=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> sudo btrfs send -v -p /mnt/sda3/autosnap/alpine-znc/20210205-080827 /mnt/sda3/autosnap/alpine-znc/20210205-080937 | pv -brtfL 0 | ssh -p 22 192.168.10.8 'sudo btrfs receive -v /mnt/vps-lxd-snapshots/alpine-znc/ '
STDERR > At subvol /mnt/sda3/autosnap/alpine-znc/20210205-080937
BTRFS_IOC_SEND returned 0
joining genl thread
120 B 0:00:00 [9.63KiB/s]
receiving snapshot 20210205-080937 uuid=a302a0f3-5ce4-8b44-86f1-429d8e56af42, ctransid=529386 parent_uuid=a302a0f3-5ce4-8b44-86f1-429d8e56af42, parent_ctransid=529386
BTRFS_IOC_SET_RECEIVED_SUBVOL uuid=a302a0f3-5ce4-8b44-86f1-429d8e56af42, stransid=529386
OUTPUT > At snapshot 20210205-080937
Remote > finished sending snapshot
Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
OUTPUT > 20210205-075748
20210205-075959
20210205-080009
20210205-080012
20210205-080016
20210205-080019
20210205-080022
20210205-080747
20210205-080749
20210205-080752
20210205-080756
20210205-080804
20210205-080807
20210205-080809
20210205-080827
20210205-080937
Remote > about to remove 1 of out of 16 backups, keeping 15
Remote > delete snapshot 20210205-080749 from host=192.168.10.8 path=/mnt/vps-lxd-snapshots/alpine-znc/
EXEC >-> ssh -p 22 192.168.10.8 sudo btrfs subvolume delete '/mnt/vps-lxd-snapshots/alpine-znc/20210205-080749'
OUTPUT > Delete subvolume (no-commit): '/mnt/vps-lxd-snapshots/alpine-znc/20210205-080749'
Remote > deleted
Local > using default value for --keep-backups: 5
Local > Parameters for cleandir: keep_backups=5, target_freespace=None, target_backups=5, max_age=None, keep_latest=False
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080752
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080752
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080752'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080752
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080809
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080809
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080809'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080809
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080022
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080022
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080022'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080022
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080016
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080016
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080016'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080016
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080009
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080009
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080009'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080009
Local > remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080804
EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-znc/20210205-080804
OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-znc/20210205-080804'
Local > done remove snapshot /mnt/sda3/autosnap/alpine-znc/20210205-080804
Local > current amount of backups: 5 have to keep a minimum of 5, stopping further deletion
[09:09:40][root@vmd58232:~]$ ls -1 /mnt/sda3/autosnap/alpine-znc/
20210205-075748
20210205-075959
20210205-080747
20210205-080827
20210205-080937BACKUP HOST (target):
root@NAS-j1900:[~]# date
Fri 05 Feb 2021 09:05:05 AM CET
root@NAS-j1900:[~]# ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
20210205-075748
20210205-075959
20210205-080009
20210205-080012
20210205-080016
20210205-080019
20210205-080022
20210205-080747
20210205-080749
20210205-080752
20210205-080756
20210205-080804
20210205-080807
20210205-080809
20210205-080827
root@NAS-j1900:[~]# ls -1 /mnt/vps-lxd-snapshots/alpine-znc/
20210205-075748
20210205-075959
20210205-080009
20210205-080012
20210205-080016
20210205-080019
20210205-080022
20210205-080747
20210205-080752
20210205-080756
20210205-080804
20210205-080807
20210205-080809
20210205-080827
20210205-080937
root@NAS-j1900:[~]# ls -1 /mnt/vps-lxd-snapshots/alpine-znc/ | wc -l
15
root@NAS-j1900:[~]# btrfs-du /mnt/vps-lxd-snapshots | grep alpine-znc
lxd/pool1/containers/VPS-alpine-znc-snap1 9.97MiB 9.97MiB 262
lxd/pool1/containers/VPS-alpine-znc-snap2 65.14MiB 65.14MiB 303
lxd/pool1/containers/alpine-znc-snap3 65.11MiB 65.11MiB 522
mnt/vps-lxd-snapshots/alpine-znc/20210205-075748 65.34MiB 16.00KiB 838
mnt/vps-lxd-snapshots/alpine-znc/20210205-075959 65.34MiB 16.00KiB 839
mnt/vps-lxd-snapshots/alpine-znc/20210205-080009 65.35MiB 36.00KiB 840
mnt/vps-lxd-snapshots/alpine-znc/20210205-080012 65.34MiB 16.00KiB 841
mnt/vps-lxd-snapshots/alpine-znc/20210205-080016 65.34MiB 16.00KiB 842
mnt/vps-lxd-snapshots/alpine-znc/20210205-080019 65.34MiB 16.00KiB 843
mnt/vps-lxd-snapshots/alpine-znc/20210205-080022 65.34MiB 16.00KiB 844
mnt/vps-lxd-snapshots/alpine-znc/20210205-080747 65.34MiB 16.00KiB 845
mnt/vps-lxd-snapshots/alpine-znc/20210205-080752 65.34MiB 16.00KiB 847
mnt/vps-lxd-snapshots/alpine-znc/20210205-080756 65.34MiB 16.00KiB 848
mnt/vps-lxd-snapshots/alpine-znc/20210205-080804 65.34MiB 16.00KiB 849
mnt/vps-lxd-snapshots/alpine-znc/20210205-080807 65.34MiB 16.00KiB 850
mnt/vps-lxd-snapshots/alpine-znc/20210205-080809 65.34MiB 16.00KiB 851
mnt/vps-lxd-snapshots/alpine-znc/20210205-080827 65.34MiB 16.00KiB 852
mnt/vps-lxd-snapshots/alpine-znc/20210205-080937 65.34MiB 16.00KiB 853 (Time is an hour late, but it doesn’t bother me because there are daily snapshots.)
CRONTAB:
0 2 * * * root /usr/local/sbin/snapbtrex.py --snap /mnt/sda3/containers/alpine-gps --path /mnt/sda3/autosnap/alpine-gps --target-backups 5 --verbose --remote-host 192.168.10.8 --remote-dir /mnt/vps-lxd-snapshots/alpine-gps --remote-keep 15 | /usr/bin/gawk '{ print strftime("[\%Y-\%m-\%d \%H:\%M:\%S]"), $0 }' >> /var/log/btrfs-repli-alpine-gps.log[11:05:13][root@vmd58232:~]$ cat /var/log/btrfs-repli-alpine-gps.log
[2021-02-05 11:05:13] 20210205-100511: Local > snapshotting path=/mnt/sda3/containers/alpine-gps to newdir=/mnt/sda3/autosnap/alpine-gps/20210205-100511
[2021-02-05 11:05:13] 20210205-100511: EXEC >-> sudo btrfs subvolume snapshot -r /mnt/sda3/containers/alpine-gps /mnt/sda3/autosnap/alpine-gps/20210205-100511
[2021-02-05 11:05:13] 20210205-100511: OUTPUT > Create a readonly snapshot of '/mnt/sda3/containers/alpine-gps' in '/mnt/sda3/autosnap/alpine-gps/20210205-100511'
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100511: Local > sync filesystem /mnt/sda3/autosnap/alpine-gps
[2021-02-05 11:05:13] 20210205-100511: EXEC >-> sudo btrfs filesystem sync /mnt/sda3/autosnap/alpine-gps
[2021-02-05 11:05:13] 20210205-100511: Local > done sync filesystem /mnt/sda3/autosnap/alpine-gps
[2021-02-05 11:05:13] 20210205-100511: Local > done snapshotting
[2021-02-05 11:05:13] 20210205-100511: Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-gps
[2021-02-05 11:05:13] 20210205-100511: EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-gps
[2021-02-05 11:05:13] 20210205-100512: OUTPUT > 20210205-075748
[2021-02-05 11:05:13] 20210205-094801
[2021-02-05 11:05:13] 20210205-095328
[2021-02-05 11:05:13] 20210205-095348
[2021-02-05 11:05:13] 20210205-095351
[2021-02-05 11:05:13] 20210205-095354
[2021-02-05 11:05:13] 20210205-095358
[2021-02-05 11:05:13] 20210205-095714
[2021-02-05 11:05:13] 20210205-095725
[2021-02-05 11:05:13] 20210205-100338
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100512: Remote > last possible parent = 20210205-100338
[2021-02-05 11:05:13] 20210205-100512: Remote > transfer: parent=20210205-100338 snap=20210205-100511
[2021-02-05 11:05:13] 20210205-100512: Remote > send snapshot from 20210205-100511 with parent 20210205-100338 to host 192.168.10.8 path=/mnt/vps-lxd-snapshots/alpine-gps
[2021-02-05 11:05:13] 20210205-100512: EXEC >-> sudo btrfs send -v -p /mnt/sda3/autosnap/alpine-gps/20210205-100338 /mnt/sda3/autosnap/alpine-gps/20210205-100511 | pv -brtfL 0 | ssh -p 22 192.168.10.8 'sudo btrfs receive -v /mnt/vps-lxd-snapshots/alpine-gps '
[2021-02-05 11:05:13] 20210205-100513: STDERR > At subvol /mnt/sda3/autosnap/alpine-gps/20210205-100511
[2021-02-05 11:05:13] BTRFS_IOC_SEND returned 0
[2021-02-05 11:05:13] joining genl thread
[2021-02-05 11:05:13] 56.9KiB 0:00:00 [ 122KiB/s]
[2021-02-05 11:05:13] receiving snapshot 20210205-100511 uuid=6b98fcb5-4797-1a4a-926c-c93318f84292, ctransid=529699 parent_uuid=32164c7e-2a30-e046-a668-065c7c35d23f, parent_ctransid=529693
[2021-02-05 11:05:13] BTRFS_IOC_SET_RECEIVED_SUBVOL uuid=6b98fcb5-4797-1a4a-926c-c93318f84292, stransid=529699
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100513: OUTPUT > At snapshot 20210205-100511
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100513: Remote > finished sending snapshot
[2021-02-05 11:05:13] 20210205-100513: Remote > list remote files host=192.168.10.8, dir=/mnt/vps-lxd-snapshots/alpine-gps
[2021-02-05 11:05:13] 20210205-100513: EXEC >-> ssh -p 22 192.168.10.8 ls -1 /mnt/vps-lxd-snapshots/alpine-gps
[2021-02-05 11:05:13] 20210205-100513: OUTPUT > 20210205-075748
[2021-02-05 11:05:13] 20210205-094801
[2021-02-05 11:05:13] 20210205-095328
[2021-02-05 11:05:13] 20210205-095348
[2021-02-05 11:05:13] 20210205-095351
[2021-02-05 11:05:13] 20210205-095354
[2021-02-05 11:05:13] 20210205-095358
[2021-02-05 11:05:13] 20210205-095714
[2021-02-05 11:05:13] 20210205-095725
[2021-02-05 11:05:13] 20210205-100338
[2021-02-05 11:05:13] 20210205-100511
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100513: Remote > No remote directories to clean, currently 11 remote backups, should keep 15
[2021-02-05 11:05:13] 20210205-100513: Local > using default value for --keep-backups: 5
[2021-02-05 11:05:13] 20210205-100513: Local > Parameters for cleandir: keep_backups=5, target_freespace=None, target_backups=5, max_age=None, keep_latest=False
[2021-02-05 11:05:13] 20210205-100513: Local > remove snapshot /mnt/sda3/autosnap/alpine-gps/20210205-095348
[2021-02-05 11:05:13] 20210205-100513: EXEC >-> sudo btrfs subvolume delete /mnt/sda3/autosnap/alpine-gps/20210205-095348
[2021-02-05 11:05:13] 20210205-100513: OUTPUT > Delete subvolume (no-commit): '/mnt/sda3/autosnap/alpine-gps/20210205-095348'
[2021-02-05 11:05:13]
[2021-02-05 11:05:13] 20210205-100513: Local > done remove snapshot /mnt/sda3/autosnap/alpine-gps/20210205-095348
[2021-02-05 11:05:13] 20210205-100513: Local > current amount of backups: 5 have to keep a minimum of 5, stopping further deletionThanks for the quick help, this script is SUPER!
Nice work!
from snapbtrex.
Related Issues (9)
- --sync-keep option not working for local transfers HOT 3
- migrate to python3
- Gracefully exit and give proper error for missing prerequisites
- Remote backup fails if last backup is not present on local system HOT 1
- snapshot time name HOT 3
- -daily/-weekly/-monthly - postfix name HOT 1
- The script has not started since the last Ubuntu update. HOT 2
- OSError: [Errno 13] Permission denied when executed as snapbtr HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snapbtrex.