Self encrypting files (convergent encryption plus obfuscation)
Crate | Travis | Appveyor | Coverage |
---|---|---|---|
| API Documentation | MaidSafe System Documention | MaidSafe web site | Safe Community site |
#Overview
A version of convergent encryption with an additional obfuscation step. This pattern allows secured data that can also be de-duplicated. This library presents an API that can be utilised in any application that provides a POSIX like filesystem interface, dealing very effectively with the content part of any data (in tests the parallelised approach can actually be faster than reading/writing data as a single stream). It is important to realise two important aspects of this library:
- This library deals with file content only
- This library provides very secure data, but does return a data structure (DataMap) that in turn requires to be secured.
![image of self encryption] (https://github.com/dirvine/self_encryption/blob/master/img/self-encryption.png?raw=true)
- Initial structure
- Test set-up
- Travis integration
- Docs creation
- Docs hosting (github.io)
- Windows CI set-up (ci.AppVeyor.com)
- Read/Write file in memory based buffer
- API version 0.0.8
- Implement disk based interface as example
- Full unit tests in lib.rs
- Integrations tests in tests module
- Benchmark tests for varying file sizes from 1 byte to 10 M/b
- API stable version 0.1.0
- Add compression pre encrypt and post encrypt in encrypt and decrypt methods
- Add task passing to allow cores to be lit up when handling chunks
- Allow any size file
- Replace sequencer with new struct and use BufferedStream to offload to disk (MemoryMapped file)
- Clean up any cache chunks when disk space is low (start + now < 90%)
- Store intermediate chunks when disk space is low (start + now < 90%)
- Add another functor to constructor to allow storage query for chunks (not get)
- Check for first last middle chunks on net and presume file stored
- Coverage analysis (coveralls ?)
- Uncomment benchmark tests read methods (require bench in beta channel or stabilised first)
#Video of the process [self_encryption process and use case video] (https://www.youtube.com/watch?v=Jnvwv4z17b4)
This library splits a file into encrypted chunks and also produces a data map for the same. This data map with encrypted chunks enables the file to be reconstituted. Instructions to use the 'basic_encryptor' example are as follows:
- Install RUST(Nightly build).
- OSX / Linux:
curl -s https://static.rust-lang.org/rustup.sh | sudo sh -s -- --channel=nightly
- Windows: Download Exe installer from http://www.rust-lang.org/install.html
- Install gcc.
- Linux:
sudo apt-get install gcc
- Windows: Any compatible gcc such as TDM-GCC
- Clone this repo / Download as zip and extract archive.
- To clone via Git:
git clone http://github.com/dirvine/self_encryption.git
- Browse to repo locally in terminal / command prompt.
cd self_encryption
- Encrypt a file:
cargo run --example basic_encryptor -- -e <full_path_to_any_file>
You should now have the example binary in ../self_encryption/target/debug/examples/
. The data_map
for the given file and it's encrypted chunks will be written to the current directory.
- Decrypt a file:
cargo run --example basic_encryptor -- -d <full_path_to_data_map> <full_destination_path_including_filename>
This will restore the original file to the given destination path.