investigate rply as an option to replace our lexer about hy HOT 4 CLOSED

parsley link: https://github.com/washort/parsley

And Allen's talk at pycon: http://pyvideo.org/video/1736/worry-free-parsers-with-parsley

parsimonious (https://github.com/erikrose/parsimonious) might be interesting, too.

from hy.

👍 thanks @willkg !

(that's actually where I got the idea from, seems neat)

parsimonius looks nice too -- add that to the to-look-at list!

from hy.

So, I took a stab at a few parser libs lately.

• Parsley: https://github.com/olasd/hy/tree/feature/parsley-parser

The grammar looks fairly clean (there are some clunky things, but overall it's not that bad), but it's DOG slow. Like, 8 times as slow. Plus it is not python3 compatible :(

• rply: https://github.com/olasd/hy/tree/feature/rply-parser

This looks even cleaner than parsley, IMHO. All the tests but a few pass.
The only real pending issue is that of getting the source position for tokens (column/line number), as rply does not give it... yet! There is a todo about that in the rply source, I'm looking at patching that in right now.
Oh, and, almost forgot, it stays quite fast (20% slowdown taking into account the need for exception handling and displaying for the failing tests, so should go down).

from hy.

What's the status? I've forked rply until @alex has a chance to review your PR.

from hy.