Optimize bzip2 decoding, if possible about kiwix-js HOT 5 CLOSED

Currently, we read the compressed file chunk by chunk, until we manage to uncompress it with bzip2. Maybe the size of a chunk could be optimized (currently 128KB)
After uncompressing it, we only keep a part of the result, corresponding to the article we're looking for.
I think the algorithm in the Qt application is more clever : they uncompress the file as as stream, and stop when they have read the entire article.
But I don't think the current bzip2 implementation can support that kind of streaming

from kiwix-js.

There seems to be another implementation worth trying : https://github.com/cscott/compressjs. It claims to be optimized and to provide decompression as a stream

from kiwix-js.

Maybe https://github.com/cscott/seek-bzip is even more suitable, as the API lets you specify the length of the output you want to have.

from kiwix-js.

I close this issue as "wontfix" : the Evopedia archive format is now deprecated

from kiwix-js.

FWIW, I wrote seek-bzip and compressjs for https://github.com/cscott/nell-wikipedia, so it's not surprising they seemed suited for your uses. ;)

from kiwix-js.