... depends on a lot of things and might be hard to compare.

just a few insights (from borg development):

• borg 1.x does some information gathering based primarily on the filename, assuming that same filename means same file.
• borg 2.0 will play a bit more on the safe side considering race conditions due to changing file systems, so it opens the file to get a file descriptor (fd) and then does the information gathering using the fd. the fd will always refer to the same fs object.
• borg >= 1.2 checks if a file has changed while it was backed up.
• these are a few reasons why more recent borg versions got a bit slower than older ones, especially on NFS, because open() and stat() are slow there.

So, sometimes speed == quick & dirty and slower == better / safer.

The less you do, the faster you get. The question is then if you still do enough / all that is needed.

it's hard to compare backup tools as some features and policies/styles will be different and hard to compare.

nevertheless, some related ideas (see feature table in the README):

Reliability | Redundant index copies

Not sure what kopia implements there, but in general, an index (or cache) is usually a data structure for quicker access to some primary/authoritative data. If the index (or cache) is lost, it can be rebuilt from the primary data.

So not sure why it needs "redundant index copies" considering the index (or cache) by definition is redundant to the primary data it is indexing (or caching).

Continue restore on bad blocks in repository

Usually there is some separate check/ repair function to bring the repository into a as-good-as-it-gets state.
But this is not necessarily done "on the fly" within a restore operation.

File management: filesystem flags missing

Some flags like immutable need yet another api (besides stat / xattrs / acls).

File management | Automatically excludes CACHEDIR.TAG(3) directories

borg can do that, but (IIRC) does not do it by default, but only if you give the cli option to enable this feature.

Remove the "automatically" part maybe?

Dedup & compression efficience | Can files be excluded from compression by extension

borg actually had code for that for a while (not sure if ever released), but it was removed again because it creates quite a lot of configuration burden on the user to manage all these extensions and what should be compressed or not. borg now has a auto,X compression mode which first tries to predict compressibility using lz4 (super fast) and then runs compressor X, if it likely is well compressible. In the other cases it uses either no compression or the already computed lz4 result, if that is shorter.

Are encryption protocols sure (AES-256-GCM / PolyChaCha / etc ) ?

You mean "secure".

What I would recommend here is checking for modern AE / AEAD (authenticated encryption [w/ associated data]) crypto.

AES-GCM, AES-OCB, Chacha20-Poly1305 are well known "ready to use" AEAD combos (borg2 will use 2nd and 3rd).

borg1 uses AES-CTR + HMAC-SHA256 (or blake2b) self-arranged combo due to its roots in / compatibility with attic backup.

Security | Can encrypted / compressed data be guessed (CRIME/BREACH style attacks)?

IIRC there is something about this in our FAQ or on the issue tracker. Result was that it is not a problem in this scenario.

Misc | Does the backup software support pre/post execution hooks?

Is this a feature or anti-feature? :-)

Anyway, for borg this feature does not make sense because it is a CLI tool. So you use either a shell script (then you can put your pre/post commands just in that script) or a wrapper or GUI (like borgmatic, vorta - then it is not borg's problem).

Hi @deajan,
Awesome work!

I took a look at your script and I have a few suggestions:

1-For backup and restore commands, please use the -threads option with 8 threads for your setup. It will significantly increase speed.

Increase -threads from 8 until you saturate the network link or see a decrease in speed.

2-During init please play with chunk size:

-chunk-size, -c the average size of chunks (default is 4M)
-max-chunk-size, -max the maximum size of chunks (default is chunk-size*4)
-min-chunk-size, -min the minimum size of chunks (default is chunk-size/4)

With homogeneous data, you should see smaller backups and better deduplication. see Chunk size details

3-Some clarifications for your shopping list on Duplicacy:

1-Redundant index copies : duplicacy doesn't use indexes. (or db)
2-Continue restore on bad blocks in repository: yes, and Erasure Coding
3-Data checksumming: yes
4-Backup mounting as filesystem: No (fuse implementation PR but not likely short term)
5-File includes / excludes bases on regexes: yes
6-Automatically excludes CACHEDIR.TAG(3) directories: No
7-Are metadatas encrypted too ?: yes
8-Can encrypted / compressed data be guessed (CRIME/BREACH style attacks)?: No
9-Can a compromised client delete backups?: No (with pub key and immutable target->requires target setup)
10-Can a compromised client restore encrypted data? No (with pub key)
11-Does the backup software support pre/post execution hooks?: yes, see Pre Command and Post Command Scripts
12-Does the backup software provide a crypto benchmark ?: there is a Benchmark command.

Important:

13- Duplicacy is serverless: Less cost, less maintenance, less attack surface..
This also means that D will always be a bit slower since it has to list before it uploads a particular chunk.
14: Duplicacy works with a ton of storage backends: Infinitely scalable and more secure.
15-No indexes or databases.

16-You should test partial restore
17-Test data should be a little bit more diverse. But I guess this is difficult
Hope this helps a bit. Feel free to join the Forum.

Keep up the good work.

Bupstash supports these, I don't know about the other tools.

There is rustic that is the successor of restic, but rustic is written in full rust language instead GO language.

• Rustic is compatible with restic backup.
• Rustic has more features than restic, I think.
• Rustic developer is a part of Restic development team and maintains Rustic and Restic. He said that Rustic generally uses less resources (CPU time and memory) than Restic.

Can you test Rustic benchmark if it is faster and more efficient than Restic?

Thanks!

Bupstash supports syncing backups to other repositories - Don't know if any other tool supports this.

Hi, I suggest to also add a config option for backupping remote source (not just remote repo as it is already done) on NFS (it may also be on the same local system). Thanks.

Quick and dirty backup tool benchmark with reproductible results

-> reproducible

Just throwing an idea out there, not sure how viable this is. Is it worth running the benchmarks two ways: (1) using default settings and (2) using settings optimized for this workload?

I ask because almost all these software have tweaks that can improve performance in certain aspects. For example, with Kopia you can use smaller chunks to enhance deduplication and thus reduce backup repository size. Kopia uses 4M chunks by default, because this is most efficient when the number of files gets large. But in a scenario where there are fewer files, changing it to 1M chunks may reduce in smaller backup repository size. I am sure the other programs have similar settings that can be tweaked.

Unless I'm missing something, Borg only supports Borg remotes - it doesn't support rclone.
You could rclone mount and try borg-ing to it, but that doesn't count and also seems to have problems: rclone/rclone#3641

I think this is an important use case

Bupstash supports incremental restores - it will only download files it detects have changed. This may be faster, I am not sure if other tools support this.

Would it be possible to do a test when backing up to cloud storage? As long as your local internet does not charge you for bandwidth, you can use Oracle Cloud Infrastructure as the target to get 10GB free storage, so that you don't have to pay for the tests, although Oracle may charge you for API calls. Scaleway offers free 75GB storage, which can be used too. Scaleway also does not charge for API calls.

I am happy to put some money in the pot to help fund this test.

You can use duplicacy init -repository source-dir ..., the .duplicacy directory is created in current directory, so it won't clobber with source-dir.

I agree the CLI of duplicacy is weird 😄