Unable to decompress data saved to disk - loading issue? about lz-string HOT 8 CLOSED

Been hitting this myself for both raw compression and uint8array compression - and if it is the same then it's an issue with NodeJS loading them again - and turning the Buffer back into usable raw data. Using any of the other (current) modes works perfectly well and has no issues, so I hope that someone sees this and has a look at it (specifically bin/cli.cjs - as soon as it works the test.sh script will have them passing, as it can save data perfectly well!)

from lz-string.

How is the grab for decompression being done? pretty much all if not all of the built in methods of reading data into the buffer sanitizes it into a "safe" string with things like control characters and such removed or escaped.

I had to write my own functions to handle this process for my projects, maybe something similar as a helper suite would be worth adding to the main library?

from lz-string.

That's pretty much the problem, the raw.bin for tattoo (test suite) is 972 bytes, if loading in NodeJS (via readFileSync) it gives a ~1092 byte (this I cant remember so well) string, but buffer.byteLength gives the correct 972 bytes - I've tried various combinations of TextDecoder and manual iterations, and can't end up with a 972 byte String to pass to stock/decode - a lot of testing (and the internal validation) shows that everything including the file writing is correct, just that re-loading step that fails.

Have you got a public repo to show how you managed it?

from lz-string.

Yeah the functions are defined here: https://github.com/JackalLabs/jackal.js/blob/v3-upgrade/src/utils/converters.ts#L100

stringToUint8Array and stringToUint16Array

the functions are used in a few places both on compressed strings and standard ones.

from lz-string.

It's going the other way around that has a problem - NodeJS loads as a Buffer (basically a Uint8array), and converting that to a string has a problem :-/

from lz-string.

I thought the on-disk saved version was the issue? anyway, this handles conversion from a buffer-like to a string.

https://github.com/JackalLabs/jackal.js/blob/v3-upgrade/src/utils/converters.ts#L70

I did some experimenting before moving on due to time crunch, but I think Node actually uses a bastardized utf. buffer <> string doesn't comply 1:1 with the behavior we see in the browser.

from lz-string.

It does something weird - I'd actually tried that one without the spread - will give that a try later when I get a chance - I've specifically told it null for encoding so it doesn't try to utf8 it, but I don't know what it's doing :-/

from lz-string.

Finally managed to figure it out - treating data as potential Uint8Array, but that it might have an odd number of characters - included here, and used for decompressing uint8array as that supports more, but encoding is a breaking change (as it now supports non-even -length data)

from lz-string.