scrabsha / expandable Goto Github PK

expandable works quite well, but it only supports a very reduced subset of Rust.

In order to make it complete, we have to write a state machine that accepts every syntactically valid Rust code (and nothing more!). This is currently done thanks to a 260 line-long invocation of the generate_grammar macro. This is very painful to understand and update properly. It certainly won't scale to the whole Rust grammar.

The other solution is to write a parser for Rust in a dsl that is then turned into a state machine by a custom compiler, and replace the generate_grammar stuff with this shiny state machine. This means more code to maintain, document and test, but this would make future development so much easier that it's worth it.

Here are the different steps we will follow:

• #30 Write a compiler that generates the state machine. The output language will be plain Rust code that is (hopefully) debuggable. The input language won't be fancy:
  • No variables.
  • No nested expressions, except blocks (needed for conditions).
  • Very limited conditionals (just a bunch of builtins).
  • Primitive loops (is it really needed?).
  • No function return value.
• #34 Modify the state machine handling logic so that it converges on transitions instead of converging on states. This will allow us to accommodate cases where the generated state machine is suboptimal and pushes unnecessary values onto its stack (otherwise, this would result in never-ending checking).
• #35 Write a parser of the Rust language in the DSL. We could take inspiration from the Ferrocene Language Specifications, the The Rust Reference, rustc_parse or even syn.
• #36 Get rid of the generate_grammar state machine, use the generated state machine instead.
• #39 Add support for the || operator in grammar_gen
• Add the whole Rust grammar to grammar.rs
  • #35
  • #40
  • #42
  • #43
  • #44,
  • #47.
• Write custom tests.
• Set up a solid test harness. For instance, we could parse code from crates.io, or the rustc test suite. weird_exprs.rs may be a challenging one :D
• ...
• Profit!

It could be interesting to benchmark the parser, but I don't consider it an important item.