Giter Club home page Giter Club logo

modern-rzip's Introduction

modern-rzip.

A backup suite. Supports FLZMA2, bzip3, LZ4, Zstandard, LSH i-node ordering deduplicating archiver, long range deduplication, encryption and recovery records. Also a refurbished partial rewrite of lrzip in active development.

Download, build and install

% git clone --recurse-submodules https://github.com/kspalaiologos/modern-rzip
% cd modern-rzip && ./configure && make -j$(nproc) common all && sudo make install

Usage

# Compress:
ar-mrzip -c data_dir | mrzip -L9 > data_dir.mar
# Decompress:
mkdir data_dir && mrzip -d < data_dir.mar | ar-mrzip -x data_dir

How it Works

modern-rzip applies a two-step process and reads file or STDIN input, passes it to the rzip pre-processor. The rzip pre-processor applies long-range redundancy reduction and then passes the streams of data to a back-end compressor. modern-rzip will, by default, test each stream with a compressibility test using lz4 prior to compression. The selected back-end compressor works on smaller data sets and ignore streams of data that may not compress well. The end result is significantly faster compression than standalone compressors and much faster decompression.

modern-rzip's compressors are:

  • Fast LZMA2 (default)
  • Zstandard
  • LZ4
  • zpaq
  • bzip3
  • rzip (pre-processing only)

modern-rzip's memory management scheme permits maximum use of system ram to pre-process files and then compress them.

Thanks

Con Kolivas - the creator of lrzip, Peter Hyman - maintainer of the lrzip-next fork.

modern-rzip's People

Contributors

areading avatar ckolivas avatar cspiegel avatar danieldjewell avatar emallickhossain avatar ghost avatar haneefmubarak avatar ib avatar irrequietus avatar jaalto avatar kata198 avatar kspalaiologos avatar lr4d avatar maeyanie avatar orthographic-pedant avatar patterner avatar pete4abw avatar zetok avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

seirdy

modern-rzip's Issues

src/mrzip.c:128:71: error: expected expression before ‘,’ token

make V=1 
make -C common all
make[1]: Entering directory '/var/tmp/portage/app-arch/modern-rzip-0.9.5/work/modern-rzip-0.9.5/common'
   CC blake2b.c
make[1]: Leaving directory '/var/tmp/portage/app-arch/modern-rzip-0.9.5/work/modern-rzip-0.9.5/common'
make mrzip
make[1]: Entering directory '/var/tmp/portage/app-arch/modern-rzip-0.9.5/work/modern-rzip-0.9.5'
   CC src/main.c
   CC src/mrzip.c
src/mrzip.c: In function ‘write_magic’:
src/mrzip.c:128:71: error: expected expression before ‘,’ token
  128 |     unsigned char magic[MAGIC_LEN] = { 'M', 'R', 'Z', 'I', MRZIP_MAJOR, MRZIP_MINOR };
      |                                                                       ^
make[1]: *** [common.mk:30: src/mrzip.o] Error 1
make[1]: Leaving directory '/var/tmp/portage/app-arch/modern-rzip-0.9.5/work/modern-rzip-0.9.5'
make: *** [Makefile:23: all] Error 2
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-pc-linux-gnu/12/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /var/tmp/portage/sys-devel/gcc-12.2.1_p20230121-r1/work/gcc-12-20230121/configure --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/gcc-bin/12 --includedir=/usr/lib/gcc/x86_64-pc-linux-gnu/12/include --datadir=/usr/share/gcc-data/x86_64-pc-linux-gnu/12 --mandir=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/man --infodir=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/info --with-gxx-include-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/12/include/g++-v12 --with-python-dir=/share/gcc-data/x86_64-pc-linux-gnu/12/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --disable-libunwind-exceptions --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 12.2.1_p20230121-r1 p10' --with-gcc-major-version-only --disable-esp --enable-libstdcxx-time --disable-libstdcxx-pch --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-multilib --with-multilib-list=m32,m64 --disable-fixed-point --enable-targets=all --enable-libgomp --disable-libssp --disable-libada --disable-cet --disable-systemtap --disable-valgrind-annotations --disable-vtable-verify --disable-libvtv --without-zstd --enable-lto --without-isl --disable-default-pie --disable-default-ssp
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 12.2.1 20230121 (Gentoo 12.2.1_p20230121-r1 p10)

Did I miss something?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.