byteverse / bytesmith Goto Github PK

The issue behind run-st was closed and run-st holds back new primitive-unlifted versions. I'm assuming you can replace the specialized version of runST w/ runST now?

decWord64 will need to use a slightly different algorithm than decWord8/16/32. Rather than checking to see if something is exceeded, we will need to check to see if the accumulator shrinks after being multiplied by 10.

Also, decWord should probably do this as well. No sense succeeding when there's overflow.

I need this function:

take :: e -> Int -> Parser e s ShortText

What it means is: take n bytes, validate that the byte sequence is a UTF-8 encoding of something, and then copy the bytes and put them in ShortText.

It is not actually a good idea to take that argument.

Currently, everything is lumped into Data.Parser.Bytes. This is growing rather unwieldy as character-parsing functions for more and more textual encodings get added. I plan introducing the following new modules:

• Data.Parser.Bytes.Ascii: rejects unicode code points above 127
• Data.Parser.Bytes.Latin: ISO/IEC 8859-1 encoding (also known as Latin1). Rejects unicode code points above 255. All byte sequences are valid representation of characters in this encoding.
• Data.Parser.Bytes.Utf8: The UTF-8 we all know and love.

There are additional possibilities for expansion: UTF-16 (big endian, little endian, or native) and UTF-32 come to mind. However, these are not important to me at the moment, so I don't have any immidiate plans to add them.

GHC makes it somewhat difficult to write platform-portable code once you start explicitly unboxing Word, Word64, etc. I need to make things portable anyway so that I can start using this as a dependency in other libraries.

It's always a pain to go back and do this retroactively, but I really need to add some tests, especially for the numeric parsers and Char parsers. Refactors are currently difficult because these are missing.

@mckeankylej writes:

orElse needs to have a infix precedence... preferably higher than <$>

I believe that "lower" was intended. Since <|> has infixl 3, I'm thinking about using this for orElse as well. This would let users use orElse anywhere that they would use <|> with another parser-combinator library. Note that <$> has infixl 4, so this would treat:

Foo <$> bar `orElse` baz

as

Foo <$> (bar `orElse` baz)

@mckeankylej Is this the behavior you're looking for?

Hi, where did Data.Bytes.unsafeIndex come from in https://github.com/andrewthad/bytesmith/blob/0076f0ca999b94e34f13616514e27f760916f953/src/Data/Bytes/Parser.hs#L227 ? It's breaking my builds and I can't find it in any package/module.

I'd like to point out that there is a bug in ghc <8 .10 that makes this code to not work correctly on Big Endian machines: https://gitlab.haskell.org/ghc/ghc/-/issues/20338

Code that uses buggy targetByteOrder function is here https://github.com/byteverse/bytesmith/blob/master/src/Data/Bytes/Parser/BigEndian.hs

@andrewthad I am sure you knew about this, since you opened https://gitlab.haskell.org/ghc/ghc/-/issues/18445 but I wanted to bring it up anyways, since I found it in github search.