v0.5.1 breaks `base64 -d` on Alpine Linux (musl libc) about base64 HOT 15 CLOSED

@aklomp, you mean I have to download that branch and compile it myself? o_O

from base64.

@sergeevabc Well yes, how else do you get a binary? This project does not publish any binaries.

But anyway, I think I'll merge the branch today because it fixes the issue, and create a follow-up issue to deal with the slow decoding. The fix works, it's pretty trivial and I tested it locally. It's just quite slow because it scans the input twice.

from base64.

Sorry, fat-fingered the post button.

from base64.

Hi, this is caused by a change in behavior of the bin/base64 utility since 0.5.0. In order to be bug-compatible with "standard" base64, it now always outputs a newline after the output. Since \n is not a valid base64 char, an error is thrown when the output is piped back into the decoder.

The fix is to run it as echo foo | ./build/bin/base64 -w0 | ./build/bin/base64 -d. The -w0 option suppresses line-buffered output.

from base64.

I hope you don’t consider this totally unexpected and confusing behaviour a feature. Since newline is not a valid character in base64, you can easily strip the trailing newline in the input to the decoder.

from base64.

The base64 utility is intended as a demo project for the base64 library, to show how to integrate the library into a codebase. I don't consider it a core feature. Before the last release, the base64 library was very simple and behaved as a dumb pipe. stdout could be piped to stdin and all was well.

But over the years a few issues have been raised (#8 is the one I could find quickly) that were caused by the fact that the behavior was not identical to the system base64. So in issue #115 I decided to fix that complaint and go for bug compatibility. So now the program always emits a trailing newline and always line-breaks the output by default. Gross, I agree, because I like dumb pipes, but it upholds the principle of least surprise.

But, and this is where you have a point, it seems like I didn't do a perfect job, because upon checking, the system base64 does seem to eat newlines, as demonstrated by:

echo foo | base64 | base64 -d

So I guess this issue is now about stripping newlines from the input.

from base64.

Possible connection to #33.

from base64.

Windows 7 x64 user here. Looking forward to test a fixed version.

from base64.

I pushed an issue126 branch which fixes this issue, but in benchmarks it's about twice as slow as the system base64's decoder on the same large input file. Probably because the algorithm is really naive and checks the input bytewise for \n.

I guess I can merge it because a working but slow program is preferable over a non-working program, but it sucks to admit defeat to the system base64 :)

from base64.

Found the speed regression. Don't write to stdout in the inner loop :) Anyway, now the timing is competitive again. Will merge soon.

from base64.

Testing base64 0.5.2 built with w64devkit on Windows 7 x64. Oh my, the executable turned out to be almost 10 times larger than the source (116658 vs 12913 bytes).

$ busybox echo -n Z2l0aHVi | base64.exe -d
github

$ busybox echo -n Z2l0aHViIAo= | base64.exe -d
github

$ busybox echo -n Z2l0aHViCg== | base64.exe -d
github

$ busybox echo -n Z2l0aHViIA0K | base64.exe -d
github

I guess the devil is in the details like spaces, carriage returns and line feeds?

from base64.

@sergeevabc I don't quite understand your comment. Are you reporting a bug?

Those strings seem to decode fine for me, including the whitespace, as shown with xxd:

$ echo -n Z2l0aHVi | bin/base64 -d | xxd
00000000: 6769 7468 7562                           github
$ echo -n Z2l0aHViIAo= | bin/base64 -d | xxd
00000000: 6769 7468 7562 200a                      github .
$ echo -n Z2l0aHViCg== | bin/base64 -d | xxd
00000000: 6769 7468 7562 0a                        github.
$ echo -n Z2l0aHViIA0K | bin/base64 -d | xxd
00000000: 6769 7468 7562 200d 0a                   github ..

About the binary size, not sure why it's so much larger, but I think static linking might have something to do with it.

from base64.

There's an issue indeed, @aklomp.

from base64.

@sergeevabc This issue has been closed. Please open a new issue if you want to file a bug report. By the way, I can't reproduce this in Linux; are you sure you are using the latest version?

from base64.

@aklomp, by the way, you should consider displaying the version so that we don't have to guess, redownload and recompile the source code. Yes, I have the latest version (0.5.2). And I suspect that issue is related to handling CRLF.

from base64.