stevebrun / reggie-the-regex-parser Goto Github PK

nfa_operations now segfault. nfa_operations must be reimplemented to account for the need for each state to have a unique identifier number.

The current model for the NFAs, States, and Transitions makes deep-copying near impossible/extremely difficult to implement. Another approach should be taken to allow for these deep copies.

Transitions should take a string with names of tokens separated by spaces in the order that they should be popped from the stack. Transitions should also take a function that'll create a token to be pushed to the stack we given the list of popped tokens.

There could be a provided function that'll push nothing to the stack.

The NFA evaluation should be able to return arbitrary data associated with terminal states as well as the length of the successful match.

You should be able to specify state transitions with the following data:

• the state to transition from
• either:
  • a string to transition over
  • an automata to transition over
• either:
  • the state to transition to
  • a function that takes the matched string transitioned over and returns:
    • what state to transition to
    • a 'token' value that can hold the data

Using this method, evaluating the automata may be able to return a list of the 'token' values generated during execution.

possible signature of adding a transition:

addTransition(
    start_state, 
    [ transition_string | transition_automata ], 
    [ token_data (*)(matched_string), ]
    destination_state);

possible signature of executing an automata:

struct { list_of_tokens, did_pass, passing_length } execute(automata);

The goal of these modifications is that the framework could in theory be used as a lexer.

Add ability to create symbols from strings to allow for O(1) pointer comparison for symbolized strings.
Back the symbol creation data structure with a trie for O(n) search.