backuppc / rsync-bpc Goto Github PK

Rsync-bpc is a customized version of rsync that is used as part of BackupPC

License: GNU General Public License v3.0

Makefile 0.27% C 77.11% M4 1.98% Shell 14.89% Perl 2.76% TeX 0.57% Python 0.14% Roff 2.24% Raku 0.04%

rsync-bpc's Introduction

WHAT IS RSYNC-BPC?
------------------

Rsync-bpc is a customized version of rsync that is used as part of
BackupPC, an open source backup system.

The main change to rsync is adding a shim layer (in the subdirectory
backuppc, and in bpc_sysCalls.c) that emulates the system calls for
accessing the file system so that rsync can directly read/write files
in BackupPC's format.

Rsync-bpc is fully line-compatible with vanilla rsync, so it can talk
to rsync servers and clients.

Rsync-bpc serves no purpose outside of BackupPC.

WHAT IS RSYNC?
--------------

Rsync is a fast and extraordinarily versatile file copying tool for
both remote and local files.

Rsync uses a delta-transfer algorithm which provides a very fast method
for bringing remote files into sync.  It does this by sending just the
differences in the files across the link, without requiring that both
sets of files are present at one of the ends of the link beforehand.  At
first glance this may seem impossible because the calculation of diffs
between two files normally requires local access to both files.

A technical report describing the rsync algorithm is included with this
package.


USAGE
-----

Basically you use rsync just like scp, but rsync has many additional
options.  To get a complete list of supported options type:

    rsync --help

See the manpage for more detailed information.


SETUP
-----

Rsync normally uses ssh or rsh for communication with remote systems.
It does not need to be setuid and requires no special privileges for
installation.  You must, however, have a working ssh or rsh system.
Using ssh is recommended for its security features.

Alternatively, rsync can run in `daemon' mode, listening on a socket.
This is generally used for public file distribution, although
authentication and access control are available.

To install rsync, first run the "configure" script.  This will create a
Makefile and config.h appropriate for your system.  Then type "make".

Note that on some systems you will have to force configure not to use
gcc because gcc may not support some features (such as 64 bit file
offsets) that your system may support.  Set the environment variable CC
to the name of your native compiler before running configure in this
case.

Once built put a copy of rsync in your search path on the local and
remote systems (or use "make install").  That's it!


RSYNC DAEMONS
-------------

Rsync can also talk to "rsync daemons" which can provide anonymous or
authenticated rsync.  See the rsyncd.conf(5) man page for details on how
to setup an rsync daemon.  See the rsync(1) man page for info on how to
connect to an rsync daemon.


WEB SITE
--------

The main rsync web site is here:

    http://rsync.samba.org/

You'll find a FAQ list, downloads, resources, HTML versions of the
manpages, etc.


MAILING LISTS
-------------

There is a mailing list for the discussion of rsync and its applications
that is open to anyone to join.  New releases are announced on this
list, and there is also an announcement-only mailing list for those that
want official announcements.  See the mailing-list page for full
details:

    http://rsync.samba.org/lists.html


BUG REPORTS
-----------

To visit this web page for full the details on bug reporting:

    http://rsync.samba.org/bugzilla.html

That page contains links to the current bug list, and information on how
to report a bug well.  You might also like to try searching the Internet
for the error message you've received, or looking in the mailing list
archives at:

    http://mail-archive.com/[email protected]/

To send a bug report, follow the instructions on the bug-tracking
page of the web site.

Alternately, email your bug report to [email protected] .


GIT REPOSITORY
--------------

If you want to get the very latest version of rsync direct from the
source code repository then you can use git:

    git clone git://git.samba.org/rsync.git

See the download page for full details on all the ways to grab the
source, including nightly tar files, web-browsing of the git repository,
etc.:

    http://rsync.samba.org/download.html


COPYRIGHT
---------

Rsync was originally written by Andrew Tridgell and is currently
maintained by Wayne Davison.   It has been improved by many developers
from around the world.

Rsync may be used, modified and redistributed only under the terms of
the GNU General Public License, found in the file COPYING in this
distribution, or at:

    http://www.fsf.org/licenses/gpl.html


AVAILABILITY
------------

The main web site for rsync is http://rsync.samba.org/
The main ftp site is ftp://rsync.samba.org/pub/rsync/
This is also available as rsync://rsync.samba.org/rsyncftp/

rsync-bpc's People

Contributors

Stargazers

Watchers

Forkers

raoulbhatia ldrolez ovidiustanila easter-eggs afiglee asilkybush seblu acidburn0zzz sun-bear jevonryan corelgott fweimer-rh machiry felag panlinux

rsync-bpc's Issues

Rewrite pool file, if pool file is zero-length

To my understanding, https://github.com/backuppc/rsync-bpc/blob/master/bpc_sysCalls.c#L784
has 2 reasons for rewriting a file

a) the file was removed from the pool
b) the file was marked for deletion and can be re-used.

I suggest to also rewrite if the pool file is zero-length. However, I am not able to understand the source code well enough to judge if chains / handling md5 collisions might play a role here.

Related to issues mentioned in backuppc/backuppc#412

Thanks,
Raoul

Possible memory leak in rsync_bpc

Looks like rsync_bpc grew to a very large size and was killed:

kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
kernel: [19879] 40001 19879 16685899 13895036   32541  2745607             0 rsync_bpc

rsync-bpc-3.1.3.0-1.el7.x86_64

Inconsistent stats at end of transfer

I had inconsistent "prior too pooling and compression" stats since using BackupPC v4, as stated in backuppc/backuppc#207

I tried to manually rerun a full sync of a host presenting inconsistent stats (107GB reported by BackupPC for first full backup when host only uses 12GB and has no sparse file.

Here's the command used and the summary:

sudo -u backuppc /usr/local/bin/rsync_bpc --bpc-top-dir /mnt/bkp/pool-01/BackupPC/ --bpc-host-name $H --bpc-share-name / --bpc-bkup-num 0 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 2 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -q\ -x\ -l\ root\ --rsync-path=nice\ -n19\ rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 --exclude=/proc --exclude=/sys --exclude=/dev --exclude=.extraswap $H:/ / 

...

Number of files: 166617
Number of files transferred: 138364
Total file size: 115539848767 bytes
Total transferred file size: 115343522111 bytes
Literal data: 138758723 bytes
Matched data: 0 bytes
File list size: 5963242
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 35428177
Total bytes received: 170458046

sent 35428177 bytes  received 170458046 bytes  112969.12 bytes/sec
total size is 115539848767  speedup is 561.18
Killed by signal 1.
DoneGen: 0 errors, 6701 filesExist, 196326656 sizeExist, 127932697 sizeExistComp, 145181 filesTotal, 115539848767 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 332784 inode

Here are the real metrics on the client host:

$ df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/rootfs               18G   10G  7.6G  57% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G  4.0K  1.9G   1% /dev/shm
tmpfs                    1.9G  193M  1.7G  11% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/vda1                497M  254M  243M  52% /boot
tmpfs                    380M     0  380M   0% /run/user/0

$ df -i
Filesystem                Inodes  IUsed    IFree IUse% Mounted on
/dev/rootfs             18317312 160391 18156921    1% /
devtmpfs                  482698    408   482290    1% /dev
tmpfs                     485251      2   485249    1% /dev/shm
tmpfs                     485251    514   484737    1% /run
tmpfs                     485251     13   485238    1% /sys/fs/cgroup
/dev/vda1                 512000    351   511649    1% /boot
tmpfs                     485251      1   485250    1% /run/user/0

As we can seen, rsync_bpc reports 107GB of data, while the server really only use 12GB.

I also noticed something strange that could explain the stats, the backup lasted way too long compared to other hosts with the same amount of data.
I suspect there could be (a large amount of) garbage / extra data transfered, that used far more bandwidth, which would explain why it was so long (about 20 times more than others, with only 2 time more data in volume).

configure does not respect rsync --disable-acl-support

If I configure rsync-bpc with --disable-acl-support, ACL is still enabled in build process leading to #27

Doing sed -i 's,lib/sysacls.o,,' Makefile* in source lead to a successful build.
Furthermore, backuppc is still working after this change.

Compiling rsync-bpc on Ubuntu 18.04

I have been trying to get rsyncd transfer working with one of our client machines, but the transfer never manages to write any data to the disk and continues eternally. Unfortunately, I cannot choose freely the Xfer method (I have sudo rights, but it's a CentOS 7 virtual server with many of our university, which has many limitations concerning policies).
After figuring out that there is rsync-bpc, I tried to compile it on Ubuntu 18.04, but get the following message:

gcc -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c lib/sysacls.c -o lib/sysacls.o
lib/sysacls.c:2761:2: error: #error No ACL functions defined for this platform!
#error No ACL functions defined for this platform!
^
Makefile:65: recipe for target 'lib/sysacls.o' failed
make: *** [lib/sysacls.o] Error 1

Any idea?

Not build for debian 10 64

Hi rsync bpc is not build for debian. for the last release, the beta works ok

the error on make is.

gcc -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c lib/sysacls.c -o lib/sysacls.o
lib/sysacls.c:2761:2: error: #error No ACL functions defined for this platform!
#error No ACL functions defined for this platform!
^~~~~
make: *** [Makefile:65: lib/sysacls.o] Error 1

Weird vanished file name

file has vanished: "/usr/local/lib/ffsroot/fusr/flocal/flib/attrib"

I am quite sure a file (or several files) has really vanished during the backup, but the logged file name is wrong. The reported file and the directory never existed on both client host and BackupPC server.

Contents of file /var/db/BackupPC/pc/mx_fsroot/XferLOG.2723, modified 2020-10-13 01:35:30 (Extracting only Errors)

XferLOG file /var/db/BackupPC/pc/mx_fsroot/XferLOG.2723 created 2020-10-13 01:17:25 

>>>skipped<<<

Running: /usr/local/bin/rsync_bpc --bpc-top-dir /var/db/BackupPC --bpc-host-name mx_fsroot --bpc-share-name fsroot --bpc-bkup-num 2723 --bpc-bkup-comp 0 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 555084 --bpc-log-level 1 --bpc-attrib-new --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 --password-file=/var/db/BackupPC/pc/mx_fsroot/.rsyncdpw98269 --exclude=/cdrom --exclude=/dev --exclude=/media --exclude=/mnt --exclude=/proc --exclude=/smb --exclude=/tmp --exclude=/usr/obj --exclude=/usr/ports --exclude=/usr/src --exclude=/var/db/portsnap --exclude=/var/tmp --exclude=/vmail --exclude=/coreland backuppc@mx::fsroot /
full backup started for directory fsroot
Xfer PIDs are now 98289
This is the rsync child about to exec /usr/local/bin/rsync_bpc
Xfer PIDs are now 98289,98290
xferPids 98289,98290
[ skipped 408 lines ]
file has vanished: "/usr/local/lib/ffsroot/fusr/flocal/flib/attrib"
[ skipped 649 lines ]
Done: 0 errors, 194 filesExist, 116904421 sizeExist, 116904421 sizeExistComp, 0 filesTotal, 0 sizeTotal, 123 filesNew, 467075837 sizeNew, 467075837 sizeNewComp, 555229 inode

>>>skipped<<<

DoneGen: 0 errors, 1 filesExist, 32 sizeExist, 32 sizeExistComp, 67806 filesTotal, 4981734188 sizeTotal, 17 filesNew, 554 sizeNew, 554 sizeNewComp, 555234 inode
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1685) [generator=3.1.3.0]
rsync_bpc exited with benign status 24 (6144)

Multiple errors after "make" rsync-3.1.3 in cygwin

 /rsync-3.1.3
$ make
gcc -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -Wno-unused-parameter  -c acls.c -o acls.o
In file included from acls.c:23:
lib/sysacls.h:118:20: error: array type has incomplete element type ‘struct acl’
  118 |         struct acl acl[1];
      |                    ^~~
acls.c: In function ‘unpack_smb_acl’:
lib/sysacls.h:108:33: error: ‘USER_OBJ’ undeclared (first use in this function); did you mean ‘ACL_USER_OBJ’?
  108 | #define SMB_ACL_USER_OBJ        USER_OBJ
      |                                 ^~~~~~~~
acls.c:286:22: note: in expansion of macro ‘SMB_ACL_USER_OBJ’
  286 |                 case SMB_ACL_USER_OBJ:
      |                      ^~~~~~~~~~~~~~~~
lib/sysacls.h:108:33: note: each undeclared identifier is reported only once for each function it appears in
  108 | #define SMB_ACL_USER_OBJ        USER_OBJ
      |                                 ^~~~~~~~
acls.c:286:22: note: in expansion of macro ‘SMB_ACL_USER_OBJ’
  286 |                 case SMB_ACL_USER_OBJ:
      |                      ^~~~~~~~~~~~~~~~

and so on... What do I do? I need to install this version?

Error: md2man routine not found

When I run ./configure, I get the following error:

Configure found the following issues:

- You need python3 and either the cmarkgfm OR commonmark python3 lib in order
  to build manpages based on the git source (manpages are included in the
  official release tar files).

even though I have python3 and commonmark installed (similarly if I install cmarkgrfm).
It seems though that the problem is that the rsync source directory is missing the exectuable md2man that is called as follows in configure.ac:

md2man_out=`"$srcdir/md2man" --test "$srcdir/rsync-ssl.1.md" 2>&1

To get config to work I needed to add --disable-md2man to the command line.
Perhaps that should be changed in the config code...

which Version to choose ?

master is based on rsync-3.1.3 ... Anno 2018
3.1.2 is based on rsync-3.1.2 ... Anno 2015
3.0.9 is based on rsync-3.0.9 ... Anno 2011

i like to go with the latest stable Version. but i dont like to have an 11 years old rsync

why is 3.0.9 the latest stable and not 3.1.2 ?
Is there any reason not to use 3.1.2 ?

thx flece

question: rsyncd localhost

I have not find backuppc chat on web like discord or other, so I try to ask here.
sorry for my simple question:

I are talking about doing backup on the same backuppc host (the server)

On backuppc 3.0 server I use for localhost backup to install rsyncd daemon and configure some share;
and all work fine.

Now on backuppc +4.0 I have to use rsync-bpc.
Is rsync-bpc comunicating with rsyncd daemon or I cannot use daemon way on the server?

Can I install rsync-bpc and install rsyncd daemon on the server and use it like on pair (server, client)

(server backuppc rsync-bpc <---> client rsyncd daemon share) pair?

forgive me I have do this question because I have not well understand how rsyncd daemon work and how is use by backuppc.

best regards,
Leonardo

lib/sysacls.c throws error when no ACLs defined

 2761 | #error No ACL functions defined for this platform!
      |  ^~~~~
make: *** [Makefile:65: lib/sysacls.o] Error 1

OK so there are no ACLs defined. How would you suggest the user corrects that please?

I'm running Ubuntu 21.10 with an existing SAMBA server supporting MacOS shares and Time Machine.

rsync_bpc: write failed on "[…]": File too large (27)

Hi,

we got the following error when trying to backup a file which has 2.049 GiB size, i.e. slightly over 2 GiB:

Running: /usr/libexec/backuppc-rsync/rsync_bpc --bpc-top-dir /var/lib/backuppc --bpc-host-name […] --bpc-share-name /home --bpc-bkup-num 520 --bpc-bkup-comp 3 --bpc-bkup-prevnum 519 --bpc-bkup-prevcomp 3 --bpc-bkup-inode0 1270854 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh\ -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 vcc-prd-01:/home/ /
full backup started for directory /home
Xfer PIDs are now 26685
This is the rsync child about to exec /usr/libexec/backuppc-rsync/rsync_bpc
Xfer PIDs are now 26685,26687
xferPids 26685,26687
    same   recv >fc........ rw-r--r--      841,     600         0 […]
rsync_bpc: write failed on "[…]": File too large (27)
Done: 0 errors, 1 filesExist, 1342177280 sizeExist, 1341849014 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 1270855 inode
rsync error: error in file IO (code 11) at receiver.c(449) [receiver=3.1.3.0]
    same   recv >fcst...... rw-r--r--        0,       0 2200378428 […]
DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 457 filesTotal, 9774730 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 1270864 inode
rsync_bpc exited with fatal status 11 (2816) (rsync error: error in file IO (code 11) at receiver.c(449) [receiver=3.1.3.0])
Xfer PIDs are now 
Got fatal error during xfer (rsync error: error in file IO (code 11) at receiver.c(449) [receiver=3.1.3.0])
Backup aborted (rsync error: error in file IO (code 11) at receiver.c(449) [receiver=3.1.3.0])

This issue is usually only present in file system related limitations. But since rsync_bpc (as I understand) doesn't write to any real file system, this is probably related to the BackupPC-specific backend.

In any case, the file system of /var/lib/backuppc/ in this case is XFS:

# xfs_info /dev/mapper/datavg0-bkpc
meta-data=/dev/mapper/datavg0-bkpc isize=512    agcount=274, agsize=8388608 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1    bigtime=0
data     =                       bsize=4096   blocks=2290676736, imaxpct=5
         =                       sunit=64     swidth=64 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=131072, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# df -hT /var/lib/backuppc/
Filesystem               Type  Size  Used Avail Use% Mounted on
/dev/mapper/datavg0-bkpc xfs   8.6T  6.5T  2.2T  76% /var/lib/backuppc

Both, the BackupPC server as well as the to be backed up host are running on Debian GNU/Linux 11 Bullseye, amd64 (x86_64) architecture.

Some more information on the used version:

# /usr/libexec/backuppc-rsync/rsync_bpc --version
rsync_bpc  version 3.1.3.0  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, no prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

'prerelease' flag for beta releases

It would be great if beta release had prerelease flag set, so that https://api.github.com/repos/backuppc/rsync-bpc/releases/latest returned latest stable version (3.1.2.2 instead of 3.1.3beta1). This way we could avoid beta version during automatic deployment.

Coverity findings

Hello, I've been asked to review rsync-bpcas part of an Ubuntu Main Inclusion Request; it's not a full audit, just a quick look. Part of this includes running the project through Coverity. I've reviewed some of the results and while there's many false positives due to the way global storage is used, I think it's found several real results.

I didn't see any SECURITY* files in the repo, and the README doesn't mention any security contacts. I don't think these issues are particularly sensitive and they may not even be part of rsync-bpc's threat model but I often lack imagination when it comes to attacks. I could post things to github issues, or email them to you, as you wish, please let me know.

Or, you could compile rsync-bpc in Coverity's free service and get all the results. I'm pretty sure their free service has the nice interface to explore results. Marking all the false positives might be a bit of work, but I think in the long run it would pay dividends by reporting bugs before users do.

Thanks

Please update to latest rsync to include security fixes

i.e.

for 3.1.x: https://download.samba.org/pub/rsync/NEWS#SECURITY_FIXES-3.1.3
for 3.2.x: https://download.samba.org/pub/rsync/NEWS#SECURITY_FIXES-3.2.5

Thanks,
Raoul

rsync-bpc fails on backuppc server with Ubuntu 22.04 LTS (Jammy Jellyfish)

rsync-bpc fails backup of PC after upgrading to Ubuntu 22.04 LTS (Jammy Jellyfish)

backuppc log for windows backup:
Error: Got fatal error during xfer (rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.3.0]) in the backuppc log.

/usr/local/bin/rsync_bpc --version
rsync_bpc  version 3.1.3.0  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, no prealloc

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

unable to make in centos 6

This is the error I am getting:

` rsync_bpc 3.1.3.beta0 configuration successful

[root@BackupPC rsync-bpc]# cd rsync-bpc/^C
[root@BackupPC rsync-bpc]# make
perl ./mkproto.pl ./*.c ./lib/compat.c
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c flist.c -o flist.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c rsync.c -o rsync.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c generator.c -o generator.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c receiver.c -o receiver.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c cleanup.c -o cleanup.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c sender.c -o sender.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c exclude.c -o exclude.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c util.c -o util.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c util2.c -o util2.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c main.c -o main.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c checksum.c -o checksum.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c match.c -o match.o
gcc -std=gnu99 -I. -I. -I./zlib -I./popt -g -O2 -DHAVE_CONFIG_H -Wall -W -c syscall.c -o syscall.o
syscall.c: In function ‘do_punch_hole’:
syscall.c:341: error: ‘FALLOC_FL_PUNCH_HOLE’ undeclared (first use in this function)
syscall.c:341: error: (Each undeclared identifier is reported only once
syscall.c:341: error: for each function it appears in.)
syscall.c:341: error: ‘FALLOC_FL_KEEP_SIZE’ undeclared (first use in this function)
make: *** [syscall.o] Error 1`

-=============-

My kernel version: Linux 2.6.32-754.9.1.el6.x86_64
Centos 6.10 64 bit minimal install

in some situations rsync stops and after some time error unexplained error (code 255) at io.c(226)

Hello,

We have a lot of servers what we are backing up via backuppc and now we have installed new backuppc system on centos 8 and noticed that on only few servers we have issues when backup stops and freezes and then gives error:

rsync_bpc: connection unexpectedly closed (42360 bytes received so far) [receiver]
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 6735 inode
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0]
[receiver] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(12)
rsync_bpc: connection unexpectedly closed (51093 bytes received so far) [generator]
DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 355 filesTotal, 15231605 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 6734 inode
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0]
[generator] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(255)

So i have started backup from cli with command /usr/share/BackupPC/bin/BackupPC_dump -f -vvv target_host and collected rsync_bpc at what rsync_bpc stucked:

/usr/bin/rsync_bpc -vvv --bpc-top-dir /data/BackupPC/ --bpc-host-name target_host --bpc-share-name /var/lib/mysql --bpc-bkup-num 1 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 6734 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh\ -p\ 11111\ -q\ -x\ -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 target_host:/var/lib/mysql/ /

And then started rcync process with this command and rsync process stucked at (Please keep in mind that this is not first directory for what backuppc was runing on same server, but last one and as i noticed backuppc for each directory generate rsync command. Also x@0024waits_global_by_latency.frm,374 is the last file from folder):

recv_generator(sys/x@0024wait_classes_global_by_avg_latency.frm,370)
recv_generator(sys/x@0024wait_classes_global_by_latency.frm,371)
recv_generator(sys/x@0024waits_by_host_by_latency.frm,372)
recv_generator(sys/x@0024waits_by_user_by_latency.frm,373)
recv_generator(sys/x@0024waits_global_by_latency.frm,374)
generate_files phase=1
rsync_bpc: connection unexpectedly closed (42360 bytes received so far) [receiver]
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 6735 inode
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0]
[receiver] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(12)
rsync_bpc: connection unexpectedly closed (51093 bytes received so far) [generator]
DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 355 filesTotal, 15231605 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 6734 inode
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0]
[generator] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(255)

Second server what has same issue:

bash-4.4$ /usr/share/BackupPC/bin/BackupPC_dump -f -vvv target2_host
Backup type: type = full, needs_full = 1, needs_incr = 1, lastFullTime = 0, opts{f} = 1, opts{i} = , opts{F} =
cmdSystemOrEval: about to system /bin/echo -c 1 target2_host
cmdSystemOrEval: about to system /bin/echo -c 1 target2_host
CheckHostAlive: can't extract round-trip time (not fatal)
CheckHostAlive: ran '/bin/echo -c 1 target2_host'; returning 0
XferLOG file /data/BackupPC//pc/target2_host/XferLOG.0.z created 2021-02-17 15:18:18
Backup prep: type = full, case = 1, inPlace = 1, doDuplicate = 0, newBkupNum = 0, newBkupIdx = 0, lastBkupNum = , lastBkupIdx = (FillCycle = 0, noFillCnt = )
bpc_progress_state backup share "/var/lib"
Running: /usr/bin/rsync_bpc --bpc-top-dir /data/BackupPC/ --bpc-host-name target2_host --bpc-share-name /var/lib --bpc-bkup-num 0 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 2 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh\ -p\ 11111\ -q\ -x\ -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 --exclude=/proc --exclude=/tmp --exclude=/var/spool/exim --exclude=/var/spool/pmta --exclude=/var/spool/postfix --exclude=*.log --exclude=*.log.* --exclude=*.pid --exclude=sess_* --exclude=*.avi --exclude=*.deb --exclude=*.dmp target2_host:/var/lib/ /
full backup started for directory /var/lib
started full dump, share=/var/lib
Xfer PIDs are now 20910
xferPids 20910
This is the rsync child about to exec /usr/bin/rsync_bpc
cmdExecOrEval: about to exec /usr/bin/rsync_bpc --bpc-top-dir /data/BackupPC/ --bpc-host-name target2+host --bpc-share-name /var/lib --bpc-bkup-num 0 --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 --bpc-bkup-inode0 2 --bpc-log-level 1 --bpc-attrib-new -e /usr/bin/ssh\ -p\ 11111\ -q\ -x\ -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 --exclude=/proc --exclude=/tmp --exclude=/var/spool/exim --exclude=/var/spool/pmta --exclude=/var/spool/postfix --exclude=*.log --exclude=*.log.* --exclude=*.pid --exclude=sess_* --exclude=*.avi --exclude=*.deb --exclude=*.dmp target2_host:/var/lib/ /
Xfer PIDs are now 20910,20913
xferPids 20910,20913
xferPids 20910,20913
new recv cd+++++++++ rwx------ 0, 0 4096 .
new recv cd+++++++++ rwxr-xr-x 0, 0 4096 1052039
bpc_progress_fileCnt 1
bpc_progress_fileCnt 21
bpc_progress_fileCnt 41
bpc_progress_fileCnt 61
bpc_progress_fileCnt 81
bpc_progress_fileCnt 101
bpc_progress_fileCnt 121
bpc_progress_fileCnt 141
same recv >f..tp..... rw-r--r-- 0, 0 0 1052039/fastboot
rsync_bpc: connection unexpectedly closed (38533 bytes received so far) [receiver]
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 7 inode
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0]
rsync_bpc: connection unexpectedly closed (54016 bytes received so far) [generator]
DoneGen: 0 errors, 159 filesExist, 3935 sizeExist, 4941 sizeExistComp, 160 filesTotal, 3935 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 2146 inode
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0]
rsync_bpc exited with fatal status 255 (65280) (rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0])
Xfer PIDs are now
xferPids
Got fatal error during xfer (rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0])
cmdSystemOrEval: about to system /bin/echo -c 1 target2_host
cmdSystemOrEval: about to system /bin/echo -c 1 target2_host
CheckHostAlive: can't extract round-trip time (not fatal)
CheckHostAlive: ran '/bin/echo -c 1 target2_host'; returning 0
Backup aborted (rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0])
bpc_progress_state fail cleanup
BackupFailCleanup: nFilesTotal = 160, type = full, BackupCase = 1, inPlace = 1, lastBkupNum =
Keeping non-empty backup #0 (/data/BackupPC//pc/target2_host/0)
bpc_progress_state fsck
Running BackupPC_refCountUpdate -h target2_host -f on target2_host
cmdSystemOrEval: about to system /usr/share/BackupPC/bin/BackupPC_refCountUpdate -h target2_host -f
Xfer PIDs are now 20922
xferPids 20922
bpc_progress_state refCnt #0
BackupPC_refCountUpdate: target2_host #0 inodeLast set to 2132 (was 1)
bpc_progress_state cntUpdate #0
bpc_progress_state rename #0
bpc_progress_state sumUpdate
bpc_progress_fileCnt 0/128
bpc_progress_fileCnt 8/128
bpc_progress_fileCnt 16/128
bpc_progress_fileCnt 24/128
bpc_progress_fileCnt 32/128
bpc_progress_fileCnt 40/128
bpc_progress_fileCnt 48/128
bpc_progress_fileCnt 56/128
bpc_progress_fileCnt 64/128
bpc_progress_fileCnt 72/128
bpc_progress_fileCnt 80/128
bpc_progress_fileCnt 88/128
bpc_progress_fileCnt 96/128
bpc_progress_fileCnt 104/128
bpc_progress_fileCnt 112/128
bpc_progress_fileCnt 120/128
bpc_progress_state rename total
BackupPC_refCountUpdate: host target2_host got 0 errors (took 0 secs)
Xfer PIDs are now
xferPids
Finished BackupPC_refCountUpdate (running time: 0 sec)
Xfer PIDs are now
xferPids
dump failed: rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.3.0]

if need to collect any additional data please tell me

bpc_poolWrite_copyToPool() appears to read and write eight bytes at a time

rsync-bpc/backuppc/bpc_poolWrite.c

Line 699 in 1b59d68

while ( (nRead = read(fdRead, info->buffer, sizeof(info->buffer))) > 0 ) {

Hello, the function bpc_poolWrite_copyToPool() appears to perform repeated read and write operations eight bytes at a time:

nt bpc_poolWrite_copyToPool(bpc_poolWrite_info *info, char *poolPath, char *fileName)
{
    int fdRead, fdWrite;
    int nRead, nWrite;

    if ( (fdWrite = open(poolPath, O_WRONLY | O_CREAT | O_EXCL, 0666)) < 0 ) {
        info->errorCnt++;
        bpc_logErrf("bpc_poolWrite_copyToPool: can't open/create %s for writing", poolPath);
        return -1;
    }
    if ( (fdRead = open(fileName, O_RDONLY)) < 0 ) {
        info->errorCnt++;
        bpc_logErrf("bpc_poolWrite_copyToPool: can't open %s for reading", fileName);
        return -1;
    }

    while ( (nRead = read(fdRead, info->buffer, sizeof(info->buffer))) > 0 ) {  /* sizeof(pointer) */
        uchar *p = info->buffer;
        int thisWrite;

        nWrite  = 0;
        while ( nWrite < nRead ) {
            do {
                thisWrite = write(fdWrite, p, nRead - nWrite);
            } while ( thisWrite < 0 && errno == EINTR );
...

(Of course on 32-bit platforms it'd be four bytes at a time, but those are dwindling.)

Depending upon how much data is being copied and how often this function is used, this could be very slow. There's a few possible approaches I can think of off the top of my head:

replace sizeof with BPC_POOL_WRITE_BUF_SZ if the buffer size is well and truly fixed
use fdopen(3) on these file descriptors to use stdio's buffered io routines
use fopen(3) on these file names to use stdio's buffered io routines (is mode 0666 correct for the output file? this seems very permissive)

Also, if the fdRead = open(fileName, ...); operation fails, fdWrite could be leaked.

Thanks

Prb with cygwin-rsyncd-3.2.3.1 under Windows 10

Hi,

I have installed cygwin-rsyncd-3.2.3.1 on my Windows 10 v20H2 and i have some issues with rsync authentication.
I have completed rsyncd.conf and rsyncd.secrets as expected.
I have completed RsyncdUserName et RsyncdPasswd on BackupPC side. Version is : 4.4.0

When i try to backup my computer on the server i have thir error :

This is the rsync child about to exec /usr/local/bin/rsync_bpc
ERROR: password file must not be other-accessible
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 2 inode
rsync error: syntax or usage error (code 1) at authenticate.c(196) [Receiver=3.1.3.0]
rsync_bpc exited with fatal status 1 (256) (rsync error: syntax or usage error (code 1) at authenticate.c(196) [Receiver=3.1.3.0])

On the computer in rsyncd.log i have :

2021/03/21 13:36:44 [211] rsyncd version 3.2.4dev-no-msgs2stderr starting, listening on port 873
2021/03/21 13:36:58 [212] name lookup failed for 192.168.1.2: Name or service not known
2021/03/21 13:36:58 [212] connect from UNKNOWN (192.168.1.2)
2021/03/21 13:36:58 [212] rsync allowed access on module users from UNKNOWN (192.168.1.2)
2021/03/21 13:36:58 [212] auth failed on module users from UNKNOWN (192.168.1.2): invalid challenge response
2021/03/21 13:37:36 [211] sent 0 bytes received 0 bytes total size 0

I don't understand the next message because i am on Winwows 10 :
This is the rsync child about to exec /usr/local/bin/rsync_bpc ERROR: password file must not be other-accessible

Any idea ?

Thank you.

Gerald.

rsync_bpc: "number": number too large or too small

Hi,

i get the following error:

backuppc@backup1:~$ /usr/bin/rsync_bpc --bpc-top-dir /backuppc/backups/ --bpc-host-name server123_hourly --bpc-share-name / --bpc-bkup-num 24669 --bpc-bkup-comp 0 --bpc-bkup-prevnum 24668 --bpc-bkup-prevcomp 0 --bpc-bkup-inode0 2147515806 --bpc-log-level 2 --bpc-attrib-new -e /usr/bin/ssh\ -q\ -x\ -l\ backupuser --rsync-path=sudo\ /usr/bin/rsync --super --recursive --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --timeout=72000 --include=/home --include=/home/backupuser --include=/home/backupuser/mysqlbackup_hourly 10.0.2.91:/ /
rsync_bpc: 2147515806: number too large or too small
rsync error: syntax or usage error (code 1) at main.c(1439) [client=3.0.9.15]

The backuppc file of the client with the jobs etc:

backuppc@backup1:~$ cat /backuppc/backups/pc/server123_hourly/backups
21993   full    1696752565      1696752934      50691   12397857860     0       0       4       981148366      0       0       0       0       0       0       981148366       0               1       rsync 04.4.0   1885302250      0       {}
21995   full    1696756156      1696756525      50691   12397834257     0       0       4       981124763      0       0       0       0       0       0       981124763       0               1       rsync 04.4.0   1885529184      0       {}
21998   full    1696759783      1696760150      50691   12397808456     0       0       4       981098962      0       0       0       0       0       0       981098962       0               1       rsync 04.4.0   1885857502      0       {}
21999   full    1696762884      1696763291      50691   12397779098     0       0       4       981069604      0       0       0       0       0       0       981069604       0               1       rsync 04.4.0   1885983052      0       {}
22000   full    1696766557      1696766932      50691   12397761486     0       0       4       981051992      0       0       0       0       0       0       981051992       0               1       rsync 04.4.0   1886108602      0       {}
22001   full    1696770290      1696770692      50691   12397769622     0       0       4       981060128      0       0       0       0       0       0       981060128       0               1       rsync 04.4.0   1886234152      0       {}
22002   full    1696773687      1696774078      50691   12397755007     0       0       4       981045513      0       0       0       0       0       0       981045513       0               1       rsync 04.4.0   1886359702      0       {}
22003   full    1696777348      1696777716      50691   12397768857     0       0       4       981059363      0       0       0       0       0       0       981059363       0               1       rsync 04.4.0   1886485252      0       {}
22004   full    1696780944      1696781315      50688   12397604736     0       0       4       981072050      0       0       0       0       0       0       981072050       0               1       rsync 04.4.0   1886610796      0       {}
22005   full    1696784547      1696784914      50688   12397617027     0       0       4       981084341      0       0       0       0       0       0       981084341       0               1       rsync 04.4.0   1886736340      0       {}
22006   full    1696788212      1696788583      50688   12397623766     0       0       4       981091080      0       0       0       0       0       0       981091080       0               1       rsync 04.4.0   1886861884      0       {}
22007   full    1696791671      1696792040      50688   12397641391     0       0       4       981108705      0       0       0       0       0       0       981108705       0               1       rsync 04.4.0   1886987428      0       {}
22008   full    1696795410      1696795790      50688   12397665100     0       0       4       981132414      0       0       0       0       0       0       981132414       0               1       rsync 04.4.0   1887112972      0       {}
22009   full    1696798872      1696799239      50688   12397686256     0       0       4       981153570      0       0       0       0       0       0       981153570       0               1       rsync 04.4.0   1887238516      0       {}
22010   full    1696802604      1696802968      50688   12397692178     0       0       4       981159492      0       0       0       0       0       0       981159492       0               1       rsync 04.4.0   1887364060      0       {}
22014   full    1696806575      1696806935      50688   12397701642     0       0       4       981168956      0       0       0       0       0       0       981168956       0               1       rsync 04.4.0   1887793738      0       {}
22017   full    1696810215      1696810573      50688   12397714204     0       0       4       981181518      0       0       0       0       0       0       981181518       0               1       rsync 04.4.0   1888122038      0       {}
22019   full    1696813766      1696814148      50688   12398436397     0       0       4       981903711      0       0       0       0       0       0       981903711       0               1       rsync 04.4.0   1888348960      0       {}
22021   full    1696817374      1696817744      50688   12401158822     0       0       4       984626136      0       0       0       0       0       0       984626136       0               1       rsync 04.4.0   1888575882      0       {}
22023   full    1696820989      1696821368      50688   12399860034     0       0       4       983327348      0       0       0       0       0       0       983327348       0               1       rsync 04.4.0   1888802804      0       {}
22025   full    1696824567      1696824933      50688   12398617365     0       0       4       982084679      0       0       0       0       0       0       982084679       0               1       rsync 04.4.0   1889029726      0       {}
22026   full    1696827671      1696828038      50688   12397633313     0       0       4       981100627      0       0       0       0       0       0       981100627       0               1       rsync 04.4.0   1889155270      0       {}
22028   full    1696831777      1696832157      50688   12397609011     0       0       4       981076325      0       0       0       0       0       0       981076325       0               1       rsync 04.4.0   1889382192      0       {}
22030   full    1696835392      1696835782      50689   12397712002     0       0       4       981184430      0       0       0       0       0       0       981184430       0               1       rsync 04.4.0   1889609118      0       {}
24435   incr    1708567201      1708567551      54130   13689493060     54124   12609544901     4     1079948159       0       0       0       0       0       12609544901     1079948159      1             1rsync   1               4.4.0   2142316755      0       {}
24436   incr    1708570801      1708571168      54130   13694010234     54124   12609544901     4     1084465333       0       0       0       0       0       12609544901     1084465333      1             1rsync   1               4.4.0   2142425017      0       {}
24437   incr    1708574401      1708574741      54130   13692738496     54124   12609544901     4     1083193595       0       0       0       0       0       12609544901     1083193595      1             1rsync   1               4.4.0   2142533279      0       {}
24438   incr    1708578001      1708578338      54130   13691327157     54124   12609544901     4     1081782256       0       0       0       0       0       12609544901     1081782256      1             1rsync   1               4.4.0   2142641541      0       {}
24439   incr    1708581601      1708581949      54130   13690073633     54124   12609544901     4     1080528732       0       0       0       0       0       12609544901     1080528732      1             1rsync   1               4.4.0   2142749803      0       {}
24440   incr    1708585201      1708585548      54131   13689447552     54124   12609544901     5     1079902651       0       0       0       0       0       12609544901     1079902651      1             1rsync   1               4.4.0   2142858067      0       {}
24441   incr    1708588801      1708589203      54133   13689566243     54125   12609560629     6     1080005614       0       0       0       0       0       12609560629     1080005614      1             1rsync   1               4.4.0   2142966335      0       {}
24442   incr    1708592400      1708592787      54141   13690396330     54127   12609599371     12    1080796959       0       0       0       0       0       12609599371     1080796959      1             1rsync   1               4.4.0   2143074619      0       {}
24443   incr    1708596001      1708596358      54137   13690372807     54130   12610093775     5     1080279032       0       0       0       0       0       12610093775     1080279032      1             1rsync   1               4.4.0   2143182895      0       {}
24444   incr    1708597801      1708598165      54138   13690591600     54131   12610112401     5     1080479199       0       0       0       0       0       12610112401     1080479199      1             1rsync   1               4.4.0   2143291173      0       {}
24445   incr    1708599601      1708600001      54139   13690652170     54132   12610248423     5     1080403747       0       0       0       0       0       12610248423     1080403747      1             1rsync   1               4.4.0   2143399453      0       {}
24446   incr    1708603631      1708604046      54140   13690745100     54135   12610267407     3     1080477693       0       0       0       0       0       12610267407     1080477693      1             1rsync   1               4.4.0   2143507735      0       {}
24447   incr    1708632001      1708632390      54158   13694912794     54125   12609955690     31    1084957104       0       0       0       0       0       12609955690     1084957104      1             1rsync   1               4.4.0   2143616053      0       {}
24448   incr    1708643161      1708643522      54158   13694929529     54152   12613602352     4     1081327177       0       0       0       0       0       12613602352     1081327177      1             1rsync   1               4.4.0   2143724371      0       {}
24449   incr    1708644601      1708644952      54158   13694938656     54152   12613602352     4     1081336304       0       0       0       0       0       12613602352     1081336304      1             1rsync   1               4.4.0   2143832689      0       {}
24450   incr    1708646401      1708646755      54158   13694918110     54152   12613602352     4     1081315758       0       0       0       0       0       12613602352     1081315758      1             1rsync   1               4.4.0   2143941007      0       {}
24451   incr    1708648519      1708648905      54158   13694915995     54153   12613603739     3     1081312256       0       0       0       0       0       12613603739     1081312256      1             1rsync   1               4.4.0   2144049325      0       {}
24452   incr    1708650002      1708650374      54158   13694923026     54152   12613602352     4     1081320674       0       0       0       0       0       12613602352     1081320674      1             1rsync   1               4.4.0   2144157643      0       {}
24453   incr    1708653601      1708653973      54158   13694982974     54152   12613602352     4     1081380622       0       0       0       0       0       12613602352     1081380622      1             1rsync   1               4.4.0   2144265961      0       {}
24454   incr    1708657201      1708657585      54158   13699445111     54152   12613602352     4     1085842759       0       0       0       0       0       12613602352     1085842759      1             1rsync   1               4.4.0   2144374279      0       {}
24455   incr    1708660801      1708661137      54158   13698129782     54152   12613602352     4     1084527430       0       0       0       0       0       12613602352     1084527430      1             1rsync   1               4.4.0   2144482597      0       {}
24456   incr    1708664401      1708664745      54158   13696688167     54152   12613602352     4     1083085815       0       0       0       0       0       12613602352     1083085815      1             1rsync   1               4.4.0   2144590915      0       {}
24457   incr    1708668001      1708668342      54158   13695502399     54152   12613602352     4     1081900047       0       0       0       0       0       12613602352     1081900047      1             1rsync   1               4.4.0   2144699233      0       {}
24458   incr    1708671601      1708671941      54158   13694938081     54152   12613602352     4     1081335729       0       0       0       0       0       12613602352     1081335729      1             1rsync   1               4.4.0   2144807551      0       {}
24459   incr    1708675201      1708675558      54160   13694998316     54152   12613602352     6     1081395964       0       0       0       0       0       12613602352     1081395964      1             1rsync   1               4.4.0   2144915873      0       {}
24460   incr    1708678800      1708679152      54160   13695109730     54154   12613611778     4     1081497952       0       0       0       0       0       12613611778     1081497952      1             1rsync   1               4.4.0   2145024195      0       {}
24461   incr    1708682401      1708682747      54162   13695231078     54154   12613611778     6     1081619300       0       0       0       0       0       12613611778     1081619300      1             1rsync   1               4.4.0   2145132521      0       {}
24462   incr    1708684201      1708684539      54162   13695315789     54156   12613627578     4     1081688211       0       0       0       0       0       12613627578     1081688211      1             1rsync   1               4.4.0   2145240847      0       {}
24463   incr    1708686001      1708686365      54162   13695386394     54156   12613627578     4     1081758816       0       0       0       0       0       12613627578     1081758816      1             1rsync   1               4.4.0   2145349173      0       {}
24464   incr    1708689601      1708689954      54162   13695455658     54156   12613627578     4     1081828080       0       0       0       0       0       12613627578     1081828080      1             1rsync   1               4.4.0   2145457499      0       {}
24465   incr    1708730015      1708730406      54164   13696147128     54159   12613668023     3     1082479105       0       0       0       0       0       12613668023     1082479105      1             1rsync   1               4.4.0   2145565829      0       {}
24466   incr    1708731001      1708731383      54164   13696149214     54158   12613666637     4     1082482577       0       0       0       0       0       12613666637     1082482577      1             1rsync   1               4.4.0   2145674159      0       {}
24467   incr    1708732801      1708733148      54164   13696121043     54158   12613666637     4     1082454406       0       0       0       0       0       12613666637     1082454406      1             1rsync   1               4.4.0   2145782489      0       {}
24468   incr    1708734601      1708734934      54164   13696127391     54158   12613666637     4     1082460754       0       0       0       0       0       12613666637     1082460754      1             1rsync   1               4.4.0   2145890819      0       {}
24469   incr    1708736401      1708736722      54164   13696127127     54158   12613666637     4     1082460490       0       0       0       0       0       12613666637     1082460490      1             1rsync   1               4.4.0   2145999149      0       {}
24470   incr    1708740001      1708740342      54164   13696189284     54158   12613666637     4     1082522647       0       0       0       0       0       12613666637     1082522647      1             1rsync   1               4.4.0   2146107479      0       {}
24471   incr    1708743601      1708743959      54164   13700699616     54158   12613666637     4     1087032979       0       0       0       0       0       12613666637     1087032979      1             1rsync   1               4.4.0   2146215809      0       {}
24472   incr    1708747201      1708747537      54164   13699372110     54158   12613666637     4     1085705473       0       0       0       0       0       12613666637     1085705473      1             1rsync   1               4.4.0   2146324139      0       {}
24473   incr    1708750800      1708751137      54164   13697947142     54158   12613666637     4     1084280505       0       0       0       0       0       12613666637     1084280505      1             1rsync   1               4.4.0   2146432469      0       {}
24474   incr    1708754400      1708754745      54165   13696745607     54158   12613666637     5     1083078970       0       0       0       0       0       12613666637     1083078970      1             1rsync   1               4.4.0   2146540801      0       {}
24475   incr    1708758001      1708758327      54165   13696158482     54159   12613685041     4     1082473441       0       0       0       0       0       12613685041     1082473441      1             1rsync   1               4.4.0   2146649133      0       {}
24476   incr    1708761601      1708761928      54165   13696130892     54159   12613685041     4     1082445851       0       0       0       0       0       12613685041     1082445851      1             1rsync   1               4.4.0   2146757465      0       {}
24477   incr    1708765201      1708765532      54165   13696120109     54159   12613685041     4     1082435068       0       0       0       0       0       12613685041     1082435068      1             1rsync   1               4.4.0   2146865797      0       {}
24478   incr    1708768801      1708769128      54165   13696105228     54159   12613685041     4     1082420187       0       0       0       0       0       12613685041     1082420187      1             1rsync   1               4.4.0   2146974129      0       {}
24479   incr    1708770601      1708770924      54165   13696138574     54159   12613685041     4     1082453533       0       0       0       0       0       12613685041     1082453533      1             1rsync   1               4.4.0   2147082461      0       {}
24480   incr    1708772401      1708772735      54165   13696115129     54159   12613685041     4     1082430088       0       0       0       0       0       12613685041     1082430088      1             1rsync   1               4.4.0   2147190793      0       {}
24481   incr    1708776338      1708776708      54165   13696083539     54160   12613686428     3     1082397111       0       0       0       0       0       12613686428     1082397111      1             1rsync   1               4.4.0   2147299125      0       {}
24482   incr    1708815961      1708816347      54169   13701322060     54153   12613605630     14    1087716430       0       0       0       0       0       12613605630     1087716430      1             1rsync   1               4.4.0   2147407465      0       {}
24669   incr    1708817401      1708817749      54169   13701327840     54163   12618836274     4     1082491566       0       0       0       0       0       12618836274     1082491566      1             1rsync   1               4.4.0   2147515805      0       {}
24670   partial 1709715600                                                                            00               1       rsync   1               4.4.0   2147515805      0

Used version of rsync_bpc:

backuppc@backup1:~$ rsync_bpc --version
rsync_bpc  version 3.1.3.0  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc

....

Also tried version 3.1.3.1 but i get the same error than:

root@backup1:~/install/rsync-bpc# rsync_bpc --version
rsync_bpc  version 3.1.3.1  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, prealloc
...

windows build

Is it possible to have a downloadable windows build?

Incomplete Element Type Error Building under Cygwin

Building on the latest version of cygwin64, I'm getting the following error. It looks like the acl structure definition is not found. This is on latest source and updated cygwin distro:

gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -Wno-unused-parameter -c acls.c -o acls.o
In file included from acls.c:23:
lib/sysacls.h:118:20: error: array type has incomplete element type ‘struct acl’
118 | struct acl acl[1];
| ^~~
make: *** [Makefile:65: acls.o] Error 1

$ gcc --version
gcc (GCC) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Why does rsync_bpc not support --max-alloc?

I recently encountered this issue:

Backup aborted (rsync error: error allocating core memory buffers (code 22) at io.c(820) [generator=3.1.3.0])

I tried to add --max-alloc=2147483648 to rsync options, and I got: rsync_bpc: --max-alloc=2147483648: unknown option

RSYNC_MAX_ALLOC env variable seems to be ignored as well.

I am running newest version of BackupPC.

backuppc 4.0.0alpha3 with rsync-bpc 3.0.9.3 and 3.0.9.4 - error in rsync protocol data stream

Hi I am using backuppc 4.0.0.alpha3 with rsync-bpc 3.0.9.3 and 3.0.9.4.
in both cases I keep getting the following error when trying to do full backup of hosts with 570 000 files.

2017-01-05 14:50:00 full backup started for directory /
2017-01-05 16:41:54 Got fatal error during xfer (rsync error: error in rsync protocol data stream (code 12) at io.c(629) [generator=3.0.9.4])
2017-01-05 16:41:59 Backup aborted (rsync error: error in rsync protocol data stream (code 12) at io.c(629) [generator=3.0.9.4])

Initially I was continuing on top of the old v3 backups, but then I removed them hoping the problem lies somewhere between the v3 vs. v4 incompatibility. Unfortunately it did not help and I keep getting the same error somewhere around 180 000 files being processed.

Warnings during build on alpine

Building rsync_bpc under alpine linux, many warnings are emitted, although the compilation finally succeeds. See the attached file:
warnings.txt

File has vanished (Similar to issue #18)

I run four BackupPC systems backing up a handful of clients each and have been getting these errors on two of the hosts for several months:

From host identified as "f-az" on client "db8-az"

incr backup started for directory /
Xfer PIDs are now 5893
This is the rsync child about to exec /usr/bin/rsync_bpc
Xfer PIDs are now 5893,5897
xferPids 5893,5897
[ skipped 83 lines ]
file has vanished: "/usr/lib/.build-id/5e/f%2f/fusr/flib/f.build-id/f5e/attrib"
file has vanished: "/usr/lib/.build-id/b3/f%2f/fusr/flib/f.build-id/fb3/attrib"
file has vanished: "/usr/lib/firmware/3com/f%2f/fusr/flib/ffirmware/f3com/attrib"
file has vanished: "/usr/lib/firmware/advansys/f%2f/fusr/flib/ffirmware/fadvansys/attrib"
file has vanished: "/usr/lib/firmware/amd-ucode/f%2f/fusr/flib/ffirmware/famd-ucode/attrib"
[ skipped 27 lines ]
Done: 0 errors, 6 filesExist, 15685422 sizeExist, 3480429 sizeExistComp, 0 filesTotal, 0 sizeTotal, 83 filesNew, 460150033 sizeNew, 371747412 sizeNewComp, 621521 inode

Number of files: 288,082 (reg: 271,192, dir: 11,347, link: 5,535, special: 8)
Number of created files: 26 (reg: 26)
Number of deleted files: 0
Number of regular files transferred: 95
Total file size: 23,859,454,943 bytes
Total transferred file size: 476,422,270 bytes
Literal data: 91,275,589 bytes
Matched data: 384,559,866 bytes
File list size: 5,199,215
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 562,483
Total bytes received: 98,890,812
sent 562,483 bytes  received 98,890,812 bytes  57,338.31 bytes/sec
total size is 23,859,454,943  speedup is 239.91
DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 276735 filesTotal, 23859454943 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 621468 inode
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1685) [generator=3.1.3.0]
rsync_bpc exited with benign status 24 (6144)
That means the client rsync had errors on some files.  Please check the XferLOG.
It likely means that rsync's delete cleanup (which deletes files on the backup
server that are no longer on the client) was skipped.  You should fix the error(s)
that rsync can run cleanly.  You can also specify the --ignore-errors option
which will still do the delete even if there are rsync errors, but do that with caution.
Xfer PIDs are now 
incr backup 213 complete, 276735 files, 23859454943 bytes, 7 xferErrs (0 bad files, 0 bad shares, 7 other)
BackupExpire: cntFull = 11, cntIncr = 6, firstFull = 0, firstIncr = 8, oldestIncr = 11.9838657407407, oldestFull = 141.025520833333
Removing filled backup 109
Xfer PIDs are now 11290
BackupPC_backupDelete: removing #109
BackupPC_backupDelete: No prior backup for merge
Xfer PIDs are now 11290,12498
BackupPC_refCountUpdate: host db8-az got 0 errors (took 48 secs)
Xfer PIDs are now 11290
Xfer PIDs are now 
Finished BackupPC_backupDelete, status = 0 (running time: 446 sec)
Xfer PIDs are now 
BackupExpire: cntFull = 10, cntIncr = 6, firstFull = 0, firstIncr = 7, oldestIncr = 11.9890277777778, oldestFull = 113.03068287037
Running BackupPC_refCountUpdate -h db8-az on db8-az
Xfer PIDs are now 12670
BackupPC_refCountUpdate: host db8-az got 0 errors (took 2 secs)
Xfer PIDs are now 
Finished BackupPC_refCountUpdate (running time: 2 sec)
Xfer PIDs are now

From host identified as "f-az" on client "o8-az"

[ skipped 15 lines ]
file has vanished: "/usr/lib/.build-id/5a/f%2f/fusr/flib/f.build-id/f5a/attrib"
file has vanished: "/usr/lib/.build-id/5c/f%2f/fusr/flib/f.build-id/f5c/attrib"
file has vanished: "/usr/lib/.build-id/5d/f%2f/fusr/flib/f.build-id/f5d/attrib"
file has vanished: "/usr/lib/.build-id/5f/f%2f/fusr/flib/f.build-id/f5f/attrib"
file has vanished: "/usr/lib/.build-id/65/f%2f/fusr/flib/f.build-id/f65/attrib"
file has vanished: "/usr/lib/.build-id/c5/f%2f/fusr/flib/f.build-id/fc5/attrib"
[ skipped 26 lines ]
Done: 0 errors, 2 filesExist, 35 sizeExist, 48 sizeExistComp, 0 filesTotal, 0 sizeTotal, 22 filesNew, 806064920 sizeNew, 777075184 sizeNewComp, 189671 inode

Number of files: 84,345 (reg: 68,007, dir: 10,882, link: 5,449, special: 7)

From host identified as "f-ev" on client "fd8-a"

file has vanished: "/etc/alternatives/f%2f/fetc/falternatives/attrib"
[ skipped 104 lines ]
Done: 0 errors, 13 filesExist, 1730988 sizeExist, 1546568 sizeExistComp, 0 filesTotal, 0 sizeTotal, 43 filesNew, 52405157 sizeNew, 18028323 sizeNewComp, 622993 inode

rsync versions on the servers:

[Cent-7:root@f-ev ~]# rpm -qa | grep -ie rsync
rsync-bpc-3.1.3.0-1.el7.x86_64
rsync-3.1.2-10.el7.x86_64

[Cent-7:user@f-az ~]$ rpm -qa | grep -ie rsync
rsync-bpc-3.1.3.0-1.el7.x86_64
rsync-3.1.2-10.el7.x86_64

rsync versions on the clients:

ssh db8-az 'rpm -qa | grep -ie rsync'
rsync-3.1.3-12.el8.x86_64

ssh o8-az 'rpm -qa | grep -ie rsync'
rsync-3.1.3-12.el8.x86_64

ssh fd8-a 'rpm -qa | grep -ie rsync'
rsync-3.1.3-12.el8.x86_64

@craigbarratt - when you suggested enabling verbose logging in issue #18 you said 'the backup seems quite small'. I'm not sure how mine compares so wanted to wait before going to the next step.
Thank you!

options.c, Message "BackupPC options are compulsory for running rsync_bpc\n" is not printed

while hunting down my problem that permissions are not correct on rsyncd restores,
I came across following error :

while playing around I noticed that the message "BackupPC options are compulsory for running rsync_bpc\n" is not printed.

options.c, Line 1880:

        if ( !bpc_top_dir || !bpc_host_name || !bpc_share_name || bpc_backup_num < 0 ) {
            snprintf(err_buf, sizeof(err_buf), "BackupPC options are compulsory for running rsync_bpc\n");
            exit_cleanup(RERR_SYNTAX);
        }

I am not a "C" Programmer, but I guess it should be :

            snprintf(err_buf, sizeof err_buf, "BackupPC options are compulsory for running rsync_bpc\n");

Does not compile on Ubuntu/aarch64

Rsync-bpc does not compile on Ubuntu 20.04.1 on Raspberry Pi 4 Model B. Seems to have an ACL problem.

Linux backup 5.4.0-1019-raspi #21-Ubuntu SMP PREEMPT Mon Sep 14 07:20:34 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

./configure --disable-acl-support does not help.

...
checking for _acl... no
checking for __acl... no
checking for _facl... no
checking for __facl... no
checking whether to support ACLs... no
...

gcc -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -Wno-unused-parameter  -c lib/sysacls.c -o lib/sysacls.o
lib/sysacls.c:2761:2: error: #error No ACL functions defined for this platform!
 2761 | #error No ACL functions defined for this platform!
      |  ^~~~~
make: *** [Makefile:65: lib/sysacls.o] Error 1

It compiles (and works!) after removing "lib/sysacls.o" from LIBOBJ in Makefile.

rsync_bpc doesn't seem to work with migrated V3 pools.

I just upgraded a V3 systems that I've been using for years. I also installed rsync_bpc. I was able to migrate all the pools to V4 without issue. I did a few backups without problems. Restores work from these new backups. However when I try to restore from one of the old backups, (using the 1. Direct rsync_bpc method) it fails. Using zip and tar method works without issues. This problem seems pretty consistent across all my nodes.

Originally, I was using an epel install of rsync_bpc (3.1.2), then I compiled the latest 3.1.3beta0 and got the same results.

Here is the logs from a simple test to restore a single file /var/www/html/logo.gif.

*** Failed restore from a migrated V3 Backup

Trimming /html from remoteDir -> /var/www/
Wrote source file list to /backup/BackupPC/pc/backupsvr/.rsyncFilesFrom6060: /html/logo.gif
Running: /usr/local/bin/rsync_bpc --bpc-top-dir /backup/BackupPC --bpc-host-name backupsvr --bpc-share-name /var/www --bpc-bkup-num 2612 --bpc-bkup-comp 7 --bpc-bkup-merge 2612/7/3 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -q\ -x\ -l\ root --rsync-path=/usr/bin/rsync --recursive --super --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --files-from=/backup/BackupPC/pc/backupsvr/.rsyncFilesFrom6060 / backupsvr:/var/www/
This is the rsync child about to exec /usr/local/bin/rsync_bpc
rsync_bpc: change_dir "/" failed: No such file or directory (2)
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 0 inode
rsync error: errors selecting input/output files, dirs (code 3) at flist.c(2100) [sender=3.1.3.beta0]
rsync_bpc exited with fatal status 3 (768) (rsync error: errors selecting input/output files, dirs (code 3) at flist.c(2100) [sender=3.1.3.beta0])
restore failed: rsync error: errors selecting input/output files, dirs (code 3) at flist.c(2100) [sender=3.1.3.beta0]

*** Successful restore from a recent V4 Backup

Trimming /html from remoteDir -> /var/www/
Wrote source file list to /backup/BackupPC/pc/backupsvr/.rsyncFilesFrom5917: /html/logo.gif
Running: /usr/local/bin/rsync_bpc --bpc-top-dir /backup/BackupPC --bpc-host-name backupsvr --bpc-share-name /var/www --bpc-bkup-num 2634 --bpc-bkup-comp 9 --bpc-bkup-merge 2634/9/4 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -q\ -x\ -l\ root --rsync-path=/usr/bin/rsync --recursive --super --protect-args --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --files-from=/backup/BackupPC/pc/backupsvr/.rsyncFilesFrom5917 / backupsvr:/var/www/
This is the rsync child about to exec /usr/local/bin/rsync_bpc
    same   send .d..t...... rwxr-xr-x        0,       0      4096 html
           send <f+++++++++ r--r--r--      501,     501      1394 html/logo.gif

Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 1,394 bytes
Total transferred file size: 1,394 bytes
Literal data: 1,394 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1,521
Total bytes received: 46
sent 1,521 bytes  received 46 bytes  1,044.67 bytes/sec
total size is 1,394  speedup is 0.89
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 1 filesTotal, 1394 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 0 inode

How to update rsync-bpc to a more recent rsync version ?

Hello,

Is it possible to update rsync-bpc to a more recent rsync version (like 3.2.7) ?

I could not understand how current rsync-bpc is created from rsync source code. Is it possible to share knowledge on this point ?

Main goal here is to continue to have backuppc in Debian, but there is a Debian bug for build backuppc-rsync package.

Note: After having seen issue #29 ; and Debian bug 1066438 report , I decided to open this issue.
Feel free to close it and tell what is appropriate way to handle this matter.

rsync_bpc segfault after upgrading to 3.0.9.9/3.0.9.10

After upgrading rsync_bpc to 3.0.9.9 (and also 3.0.9.10) I find a segfault just for one singe Solaris machine. Backup is broken for this machine.

BackupPC is 4.1.5, BackupPC-XS is 0.57.

backuppc log:

XferLOG file /var/lib/backuppc/pc/ubsun02/XferLOG.138.z created 2017-12-06 10:55:53 
Backup prep: type = incr, case = 4, inPlace = 0, doDuplicate = 0, newBkupNum = 138, newBkupIdx = 30, lastBkupNum = 137, lastBkupIdx = 29 (FillCycle = 0, noFillCnt = 1)
Running: /usr/local/bin/rsync_bpc --bpc-top-dir /var/lib/backuppc --bpc-host-name ubsun02 --bpc-share-name /etc --bpc-bkup-num 138 --bpc-bkup-comp 3 --bpc-bkup-prevnum 137 --bpc-bkup-prevcomp 3 --bpc-bkup-inode0 225396 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -oKexAlgorithms=+diffie-hellman-group1-sha1\ -l\ root --rsync-path=/usr/bin/rsync --recursive --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --timeout=72000 ubsun02:/etc/ /
incr backup started for directory /etc
Xfer PIDs are now 868
This is the rsync child about to exec /usr/local/bin/rsync_bpc
rsync_bpc exited with fatal status 0 (139) (This is the rsync child about to exec /usr/local/bin/rsync_bpc)

syslog:
Dec 6 10:55:54 backup kernel: [525132.318971] rsync_bpc[868]: segfault at 14 ip 0000000000430c47 sp 00007fff081d2ae0 error 4 in rsync_bpc[400000+7c000]

On the solaris machine

/usr/bin/rsync --recursive --numeric-ids --perms --owner --group -D --times --links --hard-links --delete --delete-excluded --one-file-system --partial --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --timeout=72000 /etc/ /tmp/test

runs without any problem.

Also no problem for any other machine (including some more solaris machines).

Directories under cpool are create with world-writable permissions

BackupPC-4 creates directories under cpool with world-writable permission (no problem in BackupPC-3).

even if these directories are below a top-level directory that is protected against traversal without proper permissions, this is bad security;
on platforms which check against world-writable files / directories, this results in security summaries with long lists of warnings;
such directories permanently pollute the backup storage of a site.

The creation of these directories has been traced to the backuppc/bpc_dirOps .c procedure of rsync-bpc (the configuration variable $Conf{UmaskMode} does not have any influence on the permission of the created directories).

This issue has relative urgency since the Mageia distro is finalising its new Mageia-7 release and would like to include BackupPC 4.3.0 in this release - see https://bugs.mageia.org/show_bug.cgi?id=24441

Sorry for the typo in the titile: create -> created