Noticed that on restoring a system that there was a significant amount of time checking the archive
and wondered if it could be quicker with parallel decompression. I know that pbzip2 has it.

tar supports a -I option to use other programs such as pbzip2

• tar -I pbzip2

possibly just reuse the -M --multi-core option for defining decompression (Restore mode).

Hi tritonas00,
thank you for this app its a nice piece of work.
I cannot see wether its an issue or maybe i'm missing something:

I'm trying to restore Arch Linux to a singled partitioned USB flash ext4. Before that all data, filesystem and signatures were wiped off the stick and then i partitioned it with parted to ext4 (without journal: mkfs.ext4 -O "^has_journal" /dev/sdXX).
Tried to restore (not transfer mode) from a btrfs and afterwards from a ext4 (out of virtualbox) running system. Tried two different backup archives and two different flash drives, with or without bootloader installation.
Here's the config file of the backup:

Auto-generated configuration file for backup mode. Place it in /etc
BRFOLDER="/run/media/kinoe/SPEICHER/BACKUPS"
BRNAME="Backup-limiarchie-20180604"
BRcompression="gzip"
BRnocolor="Yes"
BRquiet="Yes"
BRmcore="Yes"
BRthreads="2"
BRsrc="/run/media/kinoe/ARCHIE"

Do i have to place it in /etc before restoring? I didn't

Here the backup log:

[SUMMARY]
ARCHIVE
/run/media/kinoe/SPEICHER/BACKUPS/Backup-2018-06-04/Backup-limiarchie-20180604.tar.gz (/run/media/kinoe/ARCHIE) (pigz 2 threads)
ARCHIVER OPTIONS
--exclude=Backup-2018-06-04
[ARCHIVER]
tar: ARCHIE/var/lib/samba/private/msg.sock/383: Socket ignoriert
tar: ARCHIE/var/lib/samba/private/msg.sock/391: Socket ignoriert
tar: ARCHIE/var/lib/samba/private/msg.sock/390: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox-alt-1/.dropbox/iface_socket: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox-alt-1/.dropbox/command_socket: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox.old/iface_socket: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox.old/command_socket: Socket ignoriert
tar: ARCHIE/home/limi/.config/deadbeef/socket: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox/iface_socket: Socket ignoriert
tar: ARCHIE/home/limi/.dropbox/command_socket: Socket ignoriert
829913 Files archived successfully
Elapsed time: 0 hours 17 min 38 sec

Here I can see that "--threads" option, sets the maximux number of cores to use for compression. Is it possible that system-tar-and-restore, reads an input from the user to append there?

I suppose the other compression tools support something like this.

I know system-tar-and-restore Supported distributions: Arch, Debian/Ubuntu, Fedora, openSUSE, Gentoo, Mandriva/Mageia,

I want to backup a rhel5-x86-64 server, system-tar-and-restore found that bootloaders is Syslinux, but real bootloader is Grub Legacy0.97.
I could backup the rhel5-x86-64 server in "CLI" mode, but when i restore, system-tar-and-restore failed, seems like system-tar-and-restore couldn't found the real bootloader.

then how can i do?

Hello!
I try to backup system with this command:

./star.sh -i 0 -d /backup/ -c gzip -q -u "--same-permissions --same-owner --exclude=/backup/* --warning=none"

And get error at the end:

root@border1:~# ./star.sh -i 0 -d /backup/ -c gzip -q -u "--same-permissions --same-owner --exclude=/backup/* --warning=none"
System Tar & Restore 7.0
[SUMMARY]
ARCHIVE
/backup/Backup-2019-10-31/Backup-border1-20191031-163856.tar.gz (/)

ARCHIVER OPTIONS
--same-permissions
--same-owner
--exclude=/backup/*
--warning=none
--sparse
--acls
--xattrs
--exclude=/run/*
--exclude=/dev/*
--exclude=/sys/*
--exclude=/tmp/*
--exclude=/mnt/*
--exclude=/proc/*
--exclude=/media/*
--exclude=/var/run/*
--exclude=/var/lock/*
--exclude=.gvfs
--exclude=lost+found
--exclude=Backup-2019-10-31

HOME DIRECTORY
Include

FOUND BOOTLOADERS
Grub
Systemd/bootctl

[PROCESSING]
Calculating: 49521 Files
Archiving: [========================] 100%
Setting permissions to /backup/Backup-2019-10-31

An error occurred.

Check /backup/Backup-2019-10-31/backup.log for details.
Elapsed time: 0 hours 4 min 57 sec

I don't copy backup.conf, so I expect default values are used.

Log file contains extra ARCHIVE block, other part is same as log above:

[ARCHIVER]
tar: removing leading /' from member names tar: removing leading /' from hard link targets
Elapsed time: 0 hours 4 min 57 sec

Backup folder view:

root@border1:~# ls /backup/Backup-2019-10-31/ -lah
итого 991M
drwxrwxrwx 2 root root 4,0K окт 31 16:45 .
drwxr-xr-x 3 root root 4,0K окт 31 16:38 ..
-rw-rw-rw- 1 root root 991M окт 31 16:43 Backup-border1-20191031-163856.tar.gz
-rw-rw-rw- 1 root root 733 окт 31 16:43 backup.log

When I try to unpack archive I got this error and operation aborted:
Ignoring malformed pax extended attribute

What can be wrong ? Please, help!

a little UPD:
Ignoring malformed pax extended attribute appears with gui archiver in ubuntu.
tar -xzf Backup-border1-20191031-163856.tar.gz untar file without errors

OS: Ubuntu 16.04.1 Arch: Amd64 gtkdialog: 0.8.3

Gtkdialog windows can't call functions exported by star-gui.sh (scan_disks hide_used_parts set_default_pass set_default_opts set_default_multi set_args status_bar run_main).

One solution, as reported by the developer of multisystem (https://forum.ubuntu-fr.org/viewtopic.php?id=1987488), consists in putting functions in a separate script that can be sourced by gtkdialog at every function call. This does work.

This problem seems related specifically to Ubuntu 16.04. As I am not a shell expert, I have no clue about where it steams from.

I am looking for a way to add additional excludes to the existing list. I tried to edit both /etc/conf.d/system-tar-and-restore.conf and /usr/share/system-tar-and-restore/backup.example.conf but whem I try to actually compress a tar, backup.log doesn't have my options.

Of course one can always extend the defaults from the proper fields but if frequent backups have to be taken, or the exclude-list is big, it's much more convinient to "store" the program's preferences.

Either I installed the files in a wrong directory with a wrong filename, or I am missing something here.

PS: Let me know if using pastebin to show my confs will help.

Thanks a lot for sharing this code, saved me countless hours of doing stuff manually!

Hi,
First of all, many thanks for sharing your scripts.
Looking at the star.sh script, I saw that several functions were defined twice ...
Non-exhaustive list:
show_summary ()
run_calc ()
run_tar ()
The progress of the script seems to complete successfully, but it seems "dangerous".
On the other hand, I see 2 other potential problems:

1. In "Transfer" mode, hard links are not preserved by the default options of rsync: No -H option
   (no problem for "backup" mode because tar natively takes into account the hard links).
   -S (--sparse) option is missing too in default options, for copying sparse files efficiently.
2. In the "restore" mode, it seems important to provide the --numeric-owner option because of the use of an intermediate system (liveCD or liveUSB)
   see this point here http://www.gnu.org/software/tar/manual/tar.html#Attributes ; ‘--numeric-owner’
   an here : https://help.ubuntu.com/community/BackupYourSystem/TAR

and another question.

Is Live System Backup possible?

I have created an archive which is few GBs smaller than root partition. Most of its content will be placed in home which is put on separate partition. Right now I'm running out of disk space on root partition because of the archive being copied to it.

In worst case I could remove some files from the backup, but I wonder if it does sound reasonable to add a possibility to omit copying the archive - I'm reading it from external drive so I'm not concerned about performance.

Having an issue restoring an Ubuntu Server 18.04.05 installation from a live VPS to a VM on a local machine. Fully updated. Only new software are the requirements for STAR. Below is my command, its results, and my df and lsblk results if that helps.

System Tar & Restore 7.0
[INFO] UEFI environment detected. (use -W to ignore)
Checking archive type
[ERROR] /dev/sda1 is already mounted as /boot/efi, refusing to use it

df results:

System Tar & Restore 7.0
[INFO] UEFI environment detected. (use -W to ignore)
Checking archive type
[ERROR] /dev/sda1 is already mounted as /boot/efi, refusing to use it

lsblk results:

loop0    7:0    0 97.9M  1 loop /snap/core/10444
loop1    7:1    0 97.8M  1 loop /snap/core/10185
sda      8:0    0   25G  0 disk
├─sda1   8:1    0  512M  0 part /boot/efi
└─sda2   8:2    0 24.5G  0 part /

Original system df and lsblk

Filesystem         Size  Used Avail Use% Mounted on
/dev/ploop37935p1   30G   15G   14G  51% /
none               3.0G     0  3.0G   0% /sys/fs/cgroup
none               3.0G     0  3.0G   0% /dev
tmpfs              3.0G  4.0K  3.0G   1% /dev/shm
tmpfs              3.0G  1.2M  3.0G   1% /run
tmpfs              5.0M     0  5.0M   0% /run/lock
none               3.0G     0  3.0G   0% /run/shm
aaron@freder-graylog:$ lsblk
NAME           MAJ:MIN    RM SIZE RO TYPE MOUNTPOINT
ploop37935     182:606960  0  30G  0 disk
-ploop37935p1 182:606961  0  30G  0 part /

Most likely, I believe I'm misunderstanding how the boot system works in linux. But I'm unsure and haven't been able to find Google searches that match the error I see from this script.

Hello.
Just simple question.
When I made backup like this:
bin/bash star.sh -i 0 -jq -c gzip -d "/mnt/dest/backup" -n "brickbackup" -T "/"
I got that final backup was located in:

 ls -al /mnt/dest/backup/Backup-2019-11-15
total 921992
drwxrwxrwx 2 root root      4096 Nov 15 03:35 .
drwxr-xr-x 3 root root      4096 Nov 15 03:35 ..
-rw-rw-rw- 1 root root       613 Nov 15 03:44 backup.log
-rw-rw-rw- 1 root root 944100431 Nov 15 03:44 brickbackup.tar.gz

How could I create fixed destination path for backup? I mean that if I use options
-d "/mnt/dest/backup" -n "brickbackup"
I get path for created backup indeed:
/mnt/dest/backup/brickbackup.tar.gz

I have different partitions for my root (/) directory and my home dir (/home). Also I have other (Linux and data) partitions that I'd like to compress from top directory and take backup with system-tar-and-restore.

Now, only a field "Destination" is provided in order to set the store location. Could a field "Target" or something be created in order to explicitly define which directory the tarring should begin??

I know that this can be set from additional options (?) but it would be super easy and straightforward even for the noobest user.

Here is my use case:

I'm a Linux noob although experienced IT consultant with Unix experience.
I've chosen to start with Arch, struggling through as a means to learn more Linux skills.
I tend to break things later in the piece (DE/WM environments), and want to create restore points at key intervals to save time rather than completely reinstalling bits I've done 20-30 times already ;-)
system-tar-and-restore is fantastic because the CLI doesn't require a WM or DE, so I can use it on a minimal Arch install from the virtual console.

Now, as I understand it, to properly restore back to a point in time, I need to clear out the root and home partitions. Otherwise there would be "left over" files that could conflict after restoring to an earlier date.

Also, I often have other partitions mounted:
/backups is an external USB stick (where I send star.sh backups),
/boot is my UEFI partition for dual-booting Windows, (wiping this would be a disaster)
/data is a partition for "data" (VMs, media, etc), (wiping this would be a disaster)
/nfs contains a NFS share from my NAS.

After Googling a bit, this is the approach I'm using to clear out the old partitions:

umount -R /mnt  
ls /mnt  # should be empty

mount /dev/mapper/vg00-Root /mnt
mount /dev/mapper/vg00-Home /mnt/home

cd /mnt 
ls . home                           # home and other root directories should not be empty
ls boot backups data nfs  # boot, backups, data, and nfs should (must) be empty
                                 
find /mnt -mindepth 1 -delete    # wipes root and home partitions

If I have the wrong partitions accidentally mounted under root (/mnt), issuing this command could be disastrous.

I know about the -x switch, but AFAIK this will not clear out root, it merely allows the restore into a non-empty root partition.

My request is a switch that would clear out root and optionally home, would do so in a "safe" manner (from someone smarter than a Linux noob), and would cleanly restore back to a point in time as per many commercial backup/restore applications (TimeMachine, etc, etc).

Perhaps something like:

dirs="bin etc sbin usr ..."
for d in dirs; do find /mnt/$d -mindepth 1 -delete; done ???

Whatever will properly clear out root (as fast as possible) and optionally home, yet not clear out any non-standard mount points in case they were accidentally mounted. Which they shouldn't be if star.sh is controlling the mounting of the partitions and the clearing out of old files.

Thanks for listening :)

It would be nice to have multi-core support implemented while backing up the system.
Using pigz as compression program for gzip, I saved around half of the time.

Thank you very much for the script, I appreciate your work a lot.

Hello.
First of all thank You to this utility.
When I made back up from working system all works well. To restore I used Live CD Antix and again all works well.

Is there any possibility to make backup on offline system using Live CD?
To make backup I booted from live USB with Antix linux (with additional data partition for storing archives). Then mount needed root fs of my system to /mnt/source and made backup:
star.sh -i 0 -jq -c gzip -d "/mnt/archive" -n "Backup-20191114-005040" -T "/mnt/source" -g
Log after backup is that:

[SUMMARY]
ARCHIVE
/mnt/archive/Backup-2019-11-14/Backup-20191114-005040.tar.gz (/mnt/source)

ARCHIVER OPTIONS
--exclude=Backup-2019-11-14

[ARCHIVER]
tar: source/tmp/.X11-unix/X0: socket ignored
75427 Files archived successfully
Elapsed time: 0 hours 9 min 39 sec

[CONFIGURATION FILE]
# Auto-generated configuration file for backup mode. Place it in /etc

BRdestination="/mnt/archive"
BRcompression="gzip"
BRname="Backup-20191114-005040"
BRnocolor="Yes"
BRquiet="Yes"
BRsrc="/mnt/source"

I see that gzip file of backup has one level that I do not need:

root@antix1:/mnt/archive/system-tar-and-restore-master# tar -tf ../Backup-2019-11-14/Backup-20191114-005040.tar.gz  | head -n 4
source/
source/home/
source/home/proof/
source/home/proof/.lesshst
root@antix1:/mnt/archive/system-tar-and-restore-master#

Next I tried to make restore and always got an error like this
Running and target system architecture mismatch or invalid archive

For example (standard options):

root@antix1:/mnt/archive/system-tar-and-restore-master# ./star-gui.sh
star.sh -i 1 -jq -f "/mnt/archive/Backup-2019-11-14/Backup-20191114-005040.tar.gz" -r /dev/sda1 -s /dev/sda5 -G /dev/sda
System Tar & Restore 7.0
Checking archive type

[MOUNTING]
Making working directory
Mounting /dev/sda1

[BACKUP ARCHIVE]
Checking and reading archive (75427 Files)

[CLEANING AND UNMOUNTING]
Unmounting /dev/sda1
root@antix1:/mnt/archive/system-tar-and-restore-master#

I used the --strip-components=NUMBER as additional options for tar to remove one level in the path of backed archive:

root@antix1:/mnt/archive/system-tar-and-restore-master# ./star-gui.sh
star.sh -i 1 -jq -f "/mnt/archive/Backup-2019-11-14/Backup-20191114-005040.tar.gz" -u "--strip-components=1" -r /dev/sda1 -s /dev/sda5 -G /dev/sda
System Tar & Restore 7.0
Checking archive type

[MOUNTING]
Making working directory
Mounting /dev/sda1

[BACKUP ARCHIVE]
Checking and reading archive (75427 Files)

[CLEANING AND UNMOUNTING]
Unmounting /dev/sda1
root@antix1:/mnt/archive/system-tar-and-restore-master#

And with override flag:

root@antix1:/mnt/archive/system-tar-and-restore-master# ./star-gui.sh
star.sh -i 1 -jq -f "/mnt/archive/Backup-2019-11-14/Backup-20191114-005040.tar.gz" -u "--strip-components=1" -r /dev/sda1 -s /dev/sda5 -G /dev/sda -o
System Tar & Restore 7.0
Checking archive type

[MOUNTING]
Making working directory
Mounting /dev/sda1

[BACKUP ARCHIVE]
Checking and reading archive (75427 Files)

[CLEANING AND UNMOUNTING]
Unmounting /dev/sda1
root@antix1:/mnt/archive/system-tar-and-restore-master#

What should I do to execute backup and restore operations from Live CD?
Thank You

Hello!
When I specify -d parameter, this directory is auto excluded from target archive ?

I'm going to backup my ubuntu 14.04
Should I boot from the liveCD?
Or is it ok to boot from ubuntu 14.04?

/var/log/syslog doesn't working

-rw-r----- 1 sshd adm 363401 Apr 27 14:08 syslog
# chown syslog:adm syslog* (works fine after this command)
-rw-r----- 1 syslog adm 363862 May 10 17:17 syslog

why changed ownership?

Hi @tritonas00, I wrote an ebuild for Gentoo and already opened a bug in order for the Dev's to check it out. I think that a menu entry for the gui wrapper might be good.

It would require root's passwd and would also have a nice description, icon and stuff.

What do you think about it? Although I am kind of noob, I am willing to give a try and help you out with these. :)

Trying to open the archive with the default file name Backup-$(hostname)-$(date +%Y-%m-%d-%T) with tar -ztvf backup.tar.gz results in the error

tar (child): Cannot connect to Backup-localhost-19-07-2015-20: resolve failed
gzip: stdin: unexpected end of file
tar: Child returned status 128
tar: Error is not recoverable: exiting now`

After searching the internet, I ran across the following line from the GNU Tar documentation:

If the archive file name includes a colon (`:'), then it is assumed to be a file on another machine.

So basically the colon in the time format causes the error. This could be easily fixed by setting a slightly different default file name for backups, e.g. with $(date +%Y-%m-%d__%H_%M) as the date variable.

Sending this from work, not in front of my system, but hopefully my memory will suffice :)

star.sh did not work on my system which has an nvme SSD drive. IIRC the backup worked ok but the restore failed. I think it was complaining about not finding the boot partition (?).

Anyway, I grepped the script for the error msg, found the scan_part() and scan_disk() functions, and patched the script.

The solution was to clone the appropriate lines with the regex: nvme[0-9]+n[0-9]+p[0-9]+

The extent of my code review was limited to those two functions, which enabled the script to work.

My SSD partitions are named /dev/nvme0n1p1, /dev/nvme0n1p2, etc.

Happy to send more details from home if you need more info.

Thanks for the great work, much appreciated! 👍

When using star.sh with the following arguments:

#./star.sh -i 0 -d /media/user/Backup

the resulting tar file contains an incomplete home directory.

content of original home directory:

#################################################

[REDACTED]

#################################################

the content of the resulting tar file's home directory is only:

#################################################
[REDACTED]
#################################################

log:

#################################################
[REDACTED]

#################################################