Exception with odd-length array input to decompressFromUint8Array about lz-string HOT 6 CLOSED

Looking at the code, it doesn't look as if compressToUint8Array could ever produce an odd-length array.

   var buf=new Uint8Array(compressed.length*2); // 2 bytes per character
   ...
   return buf;

What it looks like is that you're not giving decompressFromUint8Array a valid output of compressToUint8Array...

Can you refine your scenario so that I can better understand how this can fail ?

from lz-string.

Basically, I'm working on a project where users have an option to download the compressed data from LZString as an octet stream. Somehow, during persistence of the stream to disk, the padding at the end of the array ends up being removed (I don't understand the mechanism by which this happens, but it does).

Reading the file back in as an ArrayBuffer works if the number of bytes happens to be even, but fails when they're odd due to the lack of padding at the end. However, if I read that same file in as a DataUri (which converts it to Base64 first and thus re-adds the padding), it works.

I can re-add the padding myself, of course, but thought it might be nice if LZString could accept arrays without the zeros at the end, for good measure.

from lz-string.

If somehow, during persistence of the stream you lose random bytes, I strongly suggest you look into this issue. It may be due to the fact that the padding occurs with a zero char, which mark the end of the file/stream/buffer in some systems. But null chars can happen anywhere in the buffer produced. Fixing my lib the way you ask is just asking me to hide your problem,

If you want to hide your problem, please do so in your own systems. Just wrap lz-string in another small object that somehow magically reconstruct what you think the input should have been. I, for one, will not fix my lib to accept an approximate version of the compressed version produced.

Come on.

from lz-string.

I don't expect an end-of-stream sequence is the problem here, but it's a fair point to require the input to decompress to match the output from compress. I'm closing this issue.

If you're ever interested in reproducing this unusual behaviour, simply persist the array output from compress by putting it in a Blob with a type of application/octet-stream, create an ObjectURL from it, and put that URL into a link with a download attribute. 50% of the time you won't be able to read it back with decompress.

from lz-string.

In which language / persistence fwk are you doing these operations ?

from lz-string.

It's vanilla JavaScript. While writing a CodePen example to demonstrate the issue, I found the cause of the missing padding.

If an array is persisted using a Blob and createObjectURL, then everything is stored verbatim as expected. If the data is persisted to disk via a DataURL, however, the browser will not store the padding since it's seen as a part of the Base64 encoding only and not part of the actual data.

Since Safari doesn't play well with ObjectURLs, I use either of these two techniques depending on the browser's support, which is why the files sometimes got their padding stripped and sometimes not.

It would be nice if LZString's array functions didn't output or require padding, but I'll leave that up to you. Your expectation for the output and input to match is perfectly reasonable. In the meantime, I can solve the issue by always reading the file as a DataURI instead of an ArrayBuffer.

from lz-string.