Hi,

I would like to try this backup program, but I don't know how to compile it.

When I run make it outputs:

make: *** No rule to make target `ClassPath/src/Utils/Dump.cpp', needed by `build/./ClassPath/Externals/ebsc/EBSC.o'.  Stop.

Can you please somekind of installation instructions.
The best if you will also provide somekind of test cases.

Regards

I might be missing something, but Frost appears to not use a rolling hash.

I setup the following test. I'm using macOS 10.12 and Frost 'Current version: 2 build 568.'

• Downloaded 'The Raven' UTF8 text from Project Gutenberg.
• Put this single file in a folder named test
• Ran Frost to backup the file to a new local folder
  Frost --backup ~/Desktop/test --remote ~/Desktop/test2

This resulted in the following chunk list (according to --dump)

  Chunk 0 with UID: 1 and offset 0
  Chunk 1 with UID: 2 and offset 6182
  Chunk 2 with UID: 3 and offset 10932
  Chunk 3 with UID: 4 and offset 17940
  Chunk 4 with UID: 5 and offset 25022
  Chunk 5 with UID: 6 and offset 27012
  Chunk 6 with UID: 7 and offset 28985
  Chunk 7 with UID: 8 and offset 33570
  Chunk 8 with UID: 9 and offset 36464
  Chunk 9 with UID: 10 and offset 39457
  Chunk 10 with UID: 11 and offset 44362
  Chunk 11 with UID: 12 and offset 46341
  Chunk 12 with UID: 13 and offset 48572
  Chunk 13 with UID: 14 and offset 51622
  Chunk 14 with UID: 15 and offset 54318
  Chunk 15 with UID: 16 and offset 60003
  Chunk 16 with UID: 17 and offset 66212

• I then modified the file by inserting 'hello world' immediately after the UTF8 BOM (offset 3), and reran Frost.

Instead of creating 1 or 2 new chunks. Frost created new chunks for the whole file again.

Notice the offsets are completely different. I expected the first few to be different, but then they should have normalized to 11 bytes different than the original and the chunk UIDs of the original should have been reused.

Frost also says both revisions are basically the same size instead of rev2 being smaller:

Revision 2 happened on Sun, 05 Nov 2017 23:43:26 GMT, linked 1 files and 1 directories, cumulative size 64.8KB (backup is 25.8KB, saved 61%)
...
Revision 1 happened on Sun, 05 Nov 2017 23:34:55 GMT, linked 1 files and 1 directories, cumulative size 64.8KB (backup is 25.7KB, saved 61%)

As I said I might be missing something interpreting the dump output, so could you explain what's going on for me?

  Chunk 0 with UID: 18 and offset 0
  Chunk 1 with UID: 19 and offset 3015
  Chunk 2 with UID: 20 and offset 5133
  Chunk 3 with UID: 21 and offset 7616
  Chunk 4 with UID: 22 and offset 10086
  Chunk 5 with UID: 23 and offset 16141
  Chunk 6 with UID: 24 and offset 18143
  Chunk 7 with UID: 25 and offset 22442
  Chunk 8 with UID: 26 and offset 24875
  Chunk 9 with UID: 27 and offset 27383
  Chunk 10 with UID: 28 and offset 30116
  Chunk 11 with UID: 29 and offset 36142
  Chunk 12 with UID: 30 and offset 38412
  Chunk 13 with UID: 31 and offset 40672
  Chunk 14 with UID: 32 and offset 45051
  Chunk 15 with UID: 33 and offset 49120
  Chunk 16 with UID: 34 and offset 51485
  Chunk 17 with UID: 35 and offset 55326
  Chunk 18 with UID: 36 and offset 59357
  Chunk 19 with UID: 37 and offset 63352
  Chunk 20 with UID: 38 and offset 66506

Thanks

Few issues I found so far:

1. For some reason it haven't restored the file:

$ ./Frost --restore ./ex/ last  --remote ./test22/  --index index_path/ --keyid 12323
Password:
Restore: ./ex/ [1/10]                                                     
Restore: ./ex/Frost 97%:70.7KB/s (rem: 662.0ms) [2/10]            Can't restore the backup: Can not decompress data from multichunk: 02AFAC197B6A61287B434483E8DACF723D371432DA0E28969E569CE8C3F234A6.#

2. It's not restoing the (chmod) access permissions.

3. For some cases it's not restoring the filesystem as it was, for example I had added files in rev. 1, then I've deleted one file in rev. 2, then I've added another file in rev 3. When I'm restoring the rev. 2 it restores with deleted file.

Hope that can help to improve software.

According to documentation Frost is using "MAC then Encrypt" scheme. But plain SHA256 is used as a MAC. Better use proper MAC, like HMAC-SHA256 keyed with a separate key derived from the master key. This will avoid leaking hashes of unencrypted data to the adversary, which could compromise data confidentiality.

Hi,

borgbackup developer here (borg is a fork of attic, with lots of fixes and some new features).

Just found your stuff, looks like we share some common goals / opinions about backup software, so it might be interesting to add borgbackup to your comparison list of backup tools.

If you want to have a look, easiest way is to use our binary build for testing.

https://github.com/borgbackup/borg/releases

Cheers, Thomas

It would be nice to have authenticated encryption for every chunks, e.g., using libsodium.
https://github.com/jedisct1/libsodium

Reference:
http://www.cryptopp.com/wiki/Authenticated_Encryption
http://en.wikipedia.org/wiki/Authenticated_encryption

Also, please take a look at s3ql encryption for each block
http://www.rath.org/s3ql-docs/impl_details.html

and tarsnap
http://www.tarsnap.com/technical.html

ClassPath/include/File/File.hpp needs ClassPath/include/Time/Timeout.hpp, which is absent, so can not compile.

./Frost --backup dir\ name --compression bsc --index index --remote remote --multichunk 10M

error: Bad argument for backup, the --backup parameter is not a folder

I have found a bug:
./Frost --backup dir --compression bsc --index index --remote remote

Compressing multichunk *** Error in `./Frost': munmap_chunk(): invalid pointer: 0x00000000040e23c0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x7198e)[0x7f2c499c698e]
/usr/lib/libc.so.6(+0x76dee)[0x7f2c499cbdee]
./Frost[0x4403ac]
./Frost[0x41a435]
./Frost[0x41c731]
./Frost[0x41ccfb]
./Frost[0x495218]
./Frost[0x4320c0]
./Frost[0x46ae6b]
./Frost[0x499691]
./Frost[0x494498]
./Frost[0x494964]
./Frost[0x4714d8]
./Frost[0x48408b]
./Frost[0x486364]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f2c49975800]
./Frost[0x405d99]
======= Memory map: ========
00400000-0054f000 r-xp 00000000 08:11 9569354 /home/trung/data/tmp/Frost
0074e000-00751000 rw-p 0014e000 08:11 9569354 /home/trung/data/tmp/Frost
00751000-00755000 rw-p 00000000 00:00 0
00c37000-0608e000 rw-p 00000000 00:00 0 [heap]
7f2c46f97000-7f2c48898000 rw-p 00000000 00:00 0
7f2c48898000-7f2c488cd000 r-xp 00000000 08:01 3432553 /usr/lib/libpng16.so.16.16.0
7f2c488cd000-7f2c48acc000 ---p 00035000 08:01 3432553 /usr/lib/libpng16.so.16.16.0
7f2c48acc000-7f2c48acd000 r--p 00034000 08:01 3432553 /usr/lib/libpng16.so.16.16.0
7f2c48acd000-7f2c48ace000 rw-p 00035000 08:01 3432553 /usr/lib/libpng16.so.16.16.0
7f2c48ace000-7f2c48add000 r-xp 00000000 08:01 3414129 /usr/lib/libbz2.so.1.0.6
7f2c48add000-7f2c48cdc000 ---p 0000f000 08:01 3414129 /usr/lib/libbz2.so.1.0.6
7f2c48cdc000-7f2c48cde000 rw-p 0000e000 08:01 3414129 /usr/lib/libbz2.so.1.0.6
7f2c48cde000-7f2c48d4c000 r-xp 00000000 08:01 3418627 /usr/lib/libpcre.so.1.2.4
7f2c48d4c000-7f2c48f4b000 ---p 0006e000 08:01 3418627 /usr/lib/libpcre.so.1.2.4
7f2c48f4b000-7f2c48f4c000 r--p 0006d000 08:01 3418627 /usr/lib/libpcre.so.1.2.4
7f2c48f4c000-7f2c48f4d000 rw-p 0006e000 08:01 3418627 /usr/lib/libpcre.so.1.2.4
7f2c48f4d000-7f2c48f62000 r-xp 00000000 08:01 3414137 /usr/lib/libz.so.1.2.8
7f2c48f62000-7f2c49161000 ---p 00015000 08:01 3414137 /usr/lib/libz.so.1.2.8
7f2c49161000-7f2c49162000 r--p 00014000 08:01 3414137 /usr/lib/libz.so.1.2.8
7f2c49162000-7f2c49163000 rw-p 00015000 08:01 3414137 /usr/lib/libz.so.1.2.8
7f2c49163000-7f2c4917f000 r-xp 00000000 08:01 3432568 /usr/lib/libgraphite2.so.3.0.1
7f2c4917f000-7f2c4937e000 ---p 0001c000 08:01 3432568 /usr/lib/libgraphite2.so.3.0.1
7f2c4937e000-7f2c49380000 r--p 0001b000 08:01 3432568 /usr/lib/libgraphite2.so.3.0.1
7f2c49380000-7f2c49381000 rw-p 0001d000 08:01 3432568 /usr/lib/libgraphite2.so.3.0.1
7f2c49381000-7f2c4943d000 r-xp 00000000 08:01 3410372 /usr/lib/libfreetype.so.6.11.4
7f2c4943d000-7f2c4963d000 ---p 000bc000 08:01 3410372 /usr/lib/libfreetype.so.6.11.4
7f2c4963d000-7f2c49643000 r--p 000bc000 08:01 3410372 /usr/lib/libfreetype.so.6.11.4
.......................

I can post the complete trace if you want, while, e.g., --multichunk 10M works.

In the backup process, right now, only a single Multichunk is opened and appended to.
I should allow the software to have two of them, one being compressed & encrypted, and the other only encrypted.
Only the chunks with a low entropy would go to the former, while the high entropy chunks will skip compression since it would not save them anything anyway.

I still get the error
./Frost --backup /path/to/dir\ name --compression bsc --index index --remote remote --multichunk 10M

error: Bad argument for backup, the --backup parameter is not a folder

Trying to setup a new backup:

$ ls -l ~/Desktop/test
total 136
-rw-r--r--  1 brian  501  66383 Dec  8 14:41 raven.txt
$ ls -l ~/Desktop/test2
ls: /Users/brian/Desktop/test2: No such file or directory
$ ./build/Frost -vv --backup ~/Desktop/test --remote ~/Desktop/test2
error: Bad argument for backup, index path does not exists

Frost (C) Copyright 2017 - Cyril RUSSO (This software is BSD licensed) 
Frost is a tool used to efficiently backup and restore files to/from a remote
place with no control other the remote server software.
No warranty of any kind is provided for the use of this software.
Current version: 3 build 579. 
...

Key store is empty:

$ ls ~/.frost/

the --backup param can only process one folder

How can I backup both /root and /opt folder in a revision