pbatard / bled Goto Github PK

View Code? Open in Web Editor NEW

30.0 9.0 4.0 535 KB

Base Library for Easy Decompression

License: GNU General Public License v2.0

Shell 6.63% C 78.69% Makefile 14.28% M4 0.40%

gzip bzip2 lzma xz zip decompression-library decompression compression-formats rufus

bled's Introduction

Bled - Base Library for Easy Decompression

What's this then?

This is Bled, the Base Library for Easy Decompression. It is based almost entirely on the native decompression code found in BusyBox.

What's it for?

It's a library, that can be used in applications to handle the decompression of .Z, .gz, .bz2, .lzma, .xz, .zip compressed files and/or archives.

Why are you doing that?

Because I need this stuff for Rufus, mostly to handle compressed disk images, and nobody outside of BusyBox seems to have put much effort providing a compact, one-size-fits-all Open Source library, for decompressing the most common formats.

As I am very conscious of size in Rufus, and Busybox is targeted at embedded systems, its decompression library seemed like a natural choice. Sure, the 7-Zip LZMA SDK is nice, and compiles nicely on Windows, but it only supports lzma/lzma2/xz (+.7z archives), so you'll need to add stuff like miniz, and then some more, and soon enough you're dealing with multiple APIs and a lot of duplicated code. Plus these libraries also provide compression support, which we don't need, so the whole thing becomes quite large.

What's the license?

GPLv2 or later.

While the BusyBox project as a whole is GPLv2 only, this library is GPLv2 or later, which means it can be used indiscriminately in GPLv2 or GPLv3 projects. This is possible because the BusyBox sources we used were all explictly tagged GPLv2 or later and any source that wasn't (libbb.h, crc32.c, decompress_unxz.c) has been recreated/replaced with versions that are. Also, since we're not using any part of bzlib there's no additional license notice required. So this really is a pure GPLv2 or later decompression library, for the most common compression formats.

What about compressed tar archives?

I'm not planning to handle these, because I have no need for tar extraction in Rufus. However, since tar archive handling is present in the BusyBox sources, I may accept a patch if you feel like adding support for it.

What about .rar?

Not planning to add support for RAR, unless a truly Free Source (GPL) RAR decompression comes along.

Are there any limitations?

Only Windows is supported for now (MinGW/gcc and MSVC). That's because:
- Windows is the only platform I need for Rufus
- The original libbb.h was GPLv2 only, so I had to recreate my own from scratch that is GPLv2 or later. This was a bit of a pain, so I don't want to bother recreating a GPLv2+ libbb.h for Linux or other platforms.
Can't query the uncompressed file size. This is mostly due to the limitation of the compression formats being used, as most of them are stream formats and do not store that information anywhere.
.7z archive extraction is not supported.
No multithreading and not optimized for speed at all!
There probably exist limitations with regards to advanced compression and/or newer compression formats.

Alright I'm sold. What do I need?

Either:

Visual Studio 2022
MinGW

Then use the .sln file or run configure + make.

Is the name a reference to Bled, Slovenia?

It is. You should visit Slovenia too, if you ever have a chance.

bled's People

Contributors

Stargazers

Watchers

Forkers

wyrover 00mjk rechie1995

bled's Issues

ER: Allow users to provide their own implementation of full_write()

See pbatard/rufus#1422 for the reasoning behind this.

Basically, similar to how users can provide their own print and progress function in bled_init(), they should also be able to provide their own implementation of full_write() (which is invoked in transformer_write and currently simply points to the _write() system call).

Error: zip method method 0 is not supported

Hello, I'm a curious college mate trying to port this library to MacOS, but when I run the test programs I get errors such as:

DECOMPRESS TO FILE: Error: zip method 0 is not supported zip: -14
OR
DECOMPRESS TO FILE: Error: zip flags 1 and 8 are not supported zip: -14
The first one was for a zipped Github repo called flyver-sdk, and the second one was an ubuntu-16.04 LTS zipped image. I've only rewritten the libbb.h and defined Windows types and system calls.

Compile as DLL

Hi,

I started using bled in a Delphi project last weekend, to do this I compiled bled as DLL in Visual Studio 2019. I just changed the type in the project configuration. If you try this there will be two linker errors. The first one is easy to fix by adding shlwapi.lib so the PathMatchSpec function can be imported. I was unable to fix the second one: LNK2019 Unresolved External Symbol _xz_crc32 in function _dec_block.
I ended up commenting the lines where xz_crc32 is used because I don't need xz support, but I think it would be nice to be able compile bled as DLL out of the box. If you like I would PR my translated Delphi header or just link a gist so it can be found by others.

ER: Use the data written rather than the data read to report progress

As per pbatard/rufus#580. This may require figuring out the expected extracted size beforehand though...

crash when extracting zip files produced on Linux

The current test/data/xz.zip was produced using 7-zip on Windows and works fine with Bled.
However, when using a xz.zip produced on Linux (using the command zip -m xz xz.h for instance), Bled crashes in decompress_gunzip.c → huft_build() due to variable d and e being NULL from the huft_build(ll, 19, 19, NULL, NULL, &inflate_codes_tl, &bl); call.

NB: This looks similar to BusyBox bug 7796, though I am doubtful BusyBox's unzip suffers from the same issue, as it would have been widely reported otherwise.

Change IOBUF_SIZE to 64K in decompress_bunzip2.c

In testing the resolution to issue #1422 in rufus, I/O performance is significantly worse when writing an image that in BZ compressed versus the straight uncompressed image. It seems that much of the performance loss is due to the [small] I/O buffer size.

Please change IOBUF_SIZE to 64K in decompress_bunzip2.c