UTF-8 conversion assist? about flawfinder HOT 4 CLOSED

Excellent. When I get a chance I intend to add that to the documentation.

from flawfinder.

Okay, I've added a brief note about character encoding to the README (pointing to the documentation), and in both places specifically noted cvt2utf and iconv (iconv was already noted in the man page). Hopefully that helps the next person!

from flawfinder.

No, because that causes other problems.

The issue isn't migrating into Python3 per se, the problem is that Python3 fails to provide useful built-in libraries to deal with code that isn't perfectly encoded.

I'm trying to ensure that flawfinder can be installed by copying a single file, since some people aren't allowed to use pip etc. to install arbitrary packages. In many circumstances that' restriction is a non-problem because of Python's "batteries included" nature. But Python3 doesn't provide a library to deal with imperfectly-encoded data, even though it added mechanisms to force perfect encoding. BeautifulSoup would certainly help, but that's not a built-in.

You don't HAVE to use UTF-8, as long as you state an encoding and stick to it.

It's very rare that people have this problem anyway, and in the long term I expect this problem to dissipate. Generally people are migrating to UTF-8 for all source files, and once that's done there is no problem.

If you do have to do a conversion, iconv may be the friend you need (as documented).

from flawfinder.

@david-a-wheeler Thanks for the response on this. As a side note, I did find https://github.com/x1angli/cvt2utf : which seems to address both our concerns?

from flawfinder.