Comments (2)
Just remembered this issue and something I'd bumped into trying to work with significant whitespace reading the Writing a custom lexer chapter in the lalrpop book. Looks like there's a way work around it (the point of that chapter) but it wasn't the way I'd hoped. I think it's a deal breaker for me, for now.
By default, LALRPOP generates a tokenizer that skips all whitespace -- including newlines. What we want is to capture whitespace characters and ignore the rest as comments, and LALRPOP does the opposite of that.
At the moment, LALRPOP doesn't allow you to configure the default tokenizer. In the future it will become quite flexible, but for now we have to write our own.
from bnf.
@CrockAgile I'll plan on taking some time to read the lalrpop
docs. I'm definitely interested potential for value it would add to this project. Keep me posted on any details.
from bnf.
Related Issues (20)
- 2 Questions about Parsing HOT 5
- Empty String Rules Fail to Match HOT 4
- Empty String Rules (Still) Fail to Match HOT 7
- faster production matching while earley parsing
- reusable grammar parser
- Nullable productions (Still, Still) Fail to Match HOT 6
- Right-recursive productions Fail to Match
- Broken coverage report HOT 6
- report the faulty character / line / column / offset when no parse trees are generated HOT 3
- How to use the bnf lib in sql parse function HOT 7
- Undefined nonterminal is accepted in bnf parsing HOT 2
- Order of Rules Affects Parsing HOT 2
- compare Criterion & Divan HOT 1
- Rethink Grammar::generate logic HOT 5
- Generate code coverage reports as a step in our GitHub actions HOT 1
- Nullable Rules?
- suggestion: extract bnf test-cases to their own .bnf files? HOT 3
- Ability to parse BNF grammars at compile time HOT 3
- Iterator::size_hint
- Include README.md as documentation in lib.rs HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bnf.