Decompression stops silently on byte #900000 about bzip2-rs HOT 4 CLOSED

Hello, speaking for the bzip2-rs crate the issue is that this file is a multistream bzip2 file, which we don't support yet. You can verify that by putting a println here and you'll see the end-of-stream is reached even though we didn't reach the end-of-file.

from bzip2-rs.

@paolobarbolini Hello, thanks for reply. I want to add it to bzip2-rs, but it seems to be a breaking change. In my view, we should remove ReadState::Eof entirely, since Decoder can never know that the end of file happened (it can only know that the end of block happened). Instead, DecoderReader should check for the end of file in the original stream (which it doesn't do now, and actually it seems to me that it can get stuck in infinite loop of feeding zero bytes to Decoder on some corrupted files which contain the header but end abruptly).

In any case, I'm more interested in pure Rust crate, so I suggest to create an issue there and continue our discussion of this there if possible.

And btw, I found a place where it is handled in the reference implementation: https://gitlab.com/federicomenaquintero/bzip2/-/blob/master/bzip2.c#L455 If you change this line to unconditional break;, bzip2 starts to give identical result to these crates.

from bzip2-rs.

I opened an issue over at my repo paolobarbolini/bzip2-rs#5 @ygyzys.

from bzip2-rs.

OK, and I'll close the issue here, because I missed that there is a MultiBzDecoder class in this crate. A bit confusing though: I don't see any reason why this should be a separate class from BzDecoder, since its functionality is strictly superior, and performance overhead for checking the end of block is hardly an issue.

from bzip2-rs.