Prelude.head empty list exception for zero or more matches. about alex HOT 5 CLOSED

Is this still an issue with the latest release?

from alex.

Yes.

$ alex Issue71.x 
alex: Prelude.head: empty list
$ alex --version
Alex version 3.2.4, (c) 2003 Chris Dornan and Simon Marlow

(2003 should be either dropped or updated.)

from alex.

The following call to head throws the error:
https://github.com/simonmar/alex/blob/f2c343b9b20ff9508f482e87528ef486c55b9f86/src/DFAMin.hs#L58

from alex.

The remaining problem is that if the lexer definition contains nullable tokens and the Alex gets stuck (on EOF or no just no possible DFA-transition), it outputs an infinite sequence of the nullable tokens. This happens because alex_scan_tkn gladly returns accepts of length 0.
The problem can be fixed by checking for the scanned length here:
https://github.com/simonmar/alex/blob/0198f7387a8f31fd8a7ff3efe453edbb00cba4fb/templates/GenericTemplate.hs#L155
and if the length is still 0, output AlexNone instead of accept or skip.

Why should a lexer not return tokens of length 0? Well, because the number of these token sequences is not determined, it could output any number of such tokens at any point. Thus, outputting 0 of such tokens only is a sensible (and the only sensible) choice.

from alex.

I'd say this issue is fixed now and can be closed.

from alex.