spinlockirqsave / rsyncme Goto Github PK
View Code? Open in Web Editor NEWA simple synchronization algorithm avoiding sending of redundant data.
License: GNU Lesser General Public License v2.1
A simple synchronization algorithm avoiding sending of redundant data.
License: GNU Lesser General Public License v2.1
/*@file README * @brief Introduction to RSYNCME. * @author Piotr Gregor <[email protected]> * @date 24 Jan 2016 06:16 PM * @copyright LGPLv2.1 */ Content 1. OVERVIEW 2. DOWNLOAD 3. DIRECTORY STRUCTURE 4. BUILD , TEST, INSTALLATION 5. TESTS 6. USAGE 7. DEBUGGING 8. CLEAN 1. OVERVIEW RSYNCME is open source tool for easy and efficient synchronization of data locally (on the same PC) and remotely (PC <-> PC) over TCP connection. The main idea is to avoid sending of redundant data, that is these bytes from file which didn't change from last sync. The bytes which need to be sent are prepared without a need of having the remote file available locally (a priori). We hope the concept to prove useful on low bandwidth links, when the time needed to transmit data overwhelms the time that alternatively will be spent by endpoints on computing differences and performing data reconstruction. The synchronization algorithm is derived from a well known rsync algorithm but we offer different options to the algorithm and believe to have better and simpler implementation. 2. DOWNLOAD Git repository: git clone --recursive https://bitbucket.org/rsyncmedev/rsyncme.git Binary packages: https://bitbucket.org/rsyncmedev/rsyncme/downloads?tab=downloads Versions: https://bitbucket.org/rsyncmedev/rsyncme/downloads?tab=tags 3. DIRECTORY STRUCTURE rsyncme/ <-- this directory (ROOT) - README This file - USER_GUIDE.txt A brief tutorial - LICENSE - include/ - twlist/ Submodule (implementation of list, queue and hashtable) - src/ - build/ - debug/ - release/ - test/ - include/ - src/ - build/ -debug/ -log/ Test logs (debug mode) are written here -release/ -log/ Test logs (release mode) are written here - install_deps.sh Script for installation of dependencies (see 4.1) /usr/ - local/ - rsyncme/ - log/ Server logs are written here 4. BUILD , TEST, INSTALLATION There are few targets available: rsyncme_d, rsyncme, test executables. Only commandline utility "rsyncme" is needed for local synchronization Installation scripts are available for out of the box installation. on Linux Debian and Ubuntu - there is no support for other platforms at this moment. Linux Debian is suggested. 4.1 Dependencies All sources depend on libuuid, so please be sure it is installed on your system or install it using our script (this will install cmocka support too, which is required if you wish to run tests). Our code depends on twlist from Piotr Gregor's github repository, so be sure you did git checkout with --recursive option: git clone --recursive https://bitbucket.org/rsyncmedev/rsyncme.git 4.1.1 Installing libuuid: A. Manually apt-get install uuid uuid-dev B. Using our script sh install_deps.sh cmocka (give name of folder where to install cmocka, e.g. "cmocka", you may need sudo to install packages) Our tests depend on cmocka, which depends on cmake, so please be sure to have them installed first if you want to run tests (you can do it manually following below instructions or run our install_deps.sh script). 4.1.2 Installing cmocka: A. Using script: sh install_deps.sh cmocka (give name of folder where to install cmocka, e.g. "cmocka", you may need sudo to install packages) B. To install manually: First, install cmake: sudo apt-get install cmake Then install cmocka: 1. Download https://cmocka.org/files/1.0/cmocka-1.0.1.tar.xz 2. Follow instructions from INSTALL file 4.2 Default release build (be sure you did git checkout with --recursive option): cd src make (will build everything [apart from tests] in release mode, same as "make all" and "make release") 4.3 Default debug build (be sure you did git checkout with --recursive option): cd src make debug 4.4 Installation make install (you may need to run this with sudo depending on your environment) 5. TESTS Please be sure dependencies are installed (see 4.1) 5.1. To build tests: Make sure cmocka support is installed (see 4.1) cd src make test ("make test-debug" for debug) 5.2 To run tests: cd src make test-check ("make test-check-debug" for debug) Make will execute all cmocka test-suites. Results of each are written in folder ROOT/test/build/release/log (ROOT/test/build/debug/log if debug tests are executed) Short summary of each executed test suite is saved in rsyncme_test_<#>_<timestamp>.log file. Detailed log is saved in rsyncme_test_<#>_<timestamp>.err for each of the test suites. If you would like to inspect intermediate files created during each test suite <#> run, please change RM_TEST_<#>_DELETE_FILES definition in header ROOT/test/include/test_rm<#>.h to 0. 6. USAGE Please read USER_GUIDE.txt. 7. DEBUGGING Debugging should be done from ROOT/test/src folder. E.g: piotr@world:~/projects/rsyncme/test/src$ gdb ../build/debug/test_rm_main1 8. CLEAN To clean everything: cd src make clean-all To clean rsyncme_d (daemon's executable) and rsyncme (commandline executable): cd src make clean To clean tests (but not test logs): cd src make test-clean To clean test logs: cd src make test-clean-logs To clean tests-debug build(but not test logs): cd src make test-clean-debug To clean test-debug logs: cd src make test-clean-logs-debug To clean test and test logs (release and debug): cd src make test-clean-all Website: http://rsyncme.org Development: https://bitbucket.org/rsyncmedev/rsyncme Mirror: https://github.com/spinlockirqsave/rsyncme Contact: [email protected]
/* @brief Copy @bytes_n bytes from @x starting at @offset
* into @dst buffer.
* @details Calls fread buffered API functions writing directly to @dst.
* File @x must be already opened.
* @return 0: success,
* -1: fseek failed,
* -2: fread failed */
int
rm_copy_buffered_2(FILE *x, size_t offset, void *dst, size_t bytes_n);
/* @brief Copy @bytes_n bytes from @x into @y.
* @details Calls fread/fwrite buffered API functions.
* Files must be already opened.
* @return 0: success,
* -1: fwrite failed,
* -2: fread failed,
* -3: other error set on @y */
int
rm_copy_buffered(FILE *x, FILE *y, size_t bytes_n);
/* @brief Copy @bytes_n bytes from @x at offset @x_offset into @y at @y_offset.
* @details Calls rm_fpread/rm_fpwrite buffered API functions.
* Files must be already opened.
* @return 0: success,
* -1: rm_fpwrite failed,
* -2: rm_fpread failed,
* -3: other error set on @y */
int
rm_copy_buffered_offset(FILE *x, FILE *y, size_t bytes_n, size_t x_offset, size_t y_offset);
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.