Comments (5)
I modified my parser manually and was able to prevent the errors from being thrown. It appears that the inputStream
might be the tokenStream
, so I attached that to the parser. The ATN states don't have a getTransitions()
method on them, even though c3 expects to always retrieve transitions this way, so I mapped a getTransitions
method onto each state which returns the state.transitions
that are defined.
const suggestExpression = (input) => {
const inputStream = new antlr4.InputStream(input);
const lexer = new ExprLexer(inputStream);
const tokenStream = new antlr4.CommonTokenStream(lexer);
const parser = new ExprParser(tokenStream);
const errorListener = new antlr4.error.ErrorListener();
parser.addErrorListener(errorListener);
parser.inputStream = tokenStream;
parser.atn.states.forEach((state) => {
state.getTransitions = () => state.transitions;
});
const tree = parser.expression();
const core = new c3.CodeCompletionCore(parser);
return core.collectCandidates(input - 1);
};
I think this is a bit of progress, however I'm still getting 0 candidates back. I assume this has to do with the lack of CodeCompletionCore.preferredRules
defined, but the test suggests otherwise:
let inputStream = new ANTLRInputStream("var c = a + b()");
let lexer = new ExprLexer(inputStream);
let tokenStream = new CommonTokenStream(lexer);
let parser = new ExprParser(tokenStream);
let errorListener = new ErrorListener();
parser.addErrorListener(errorListener);
let tree = parser.expression();
expect(errorListener.errorCount, "Test 1").equals(0);
let core = new c3.CodeCompletionCore(parser);
// 1) At the input start.
let candidates = core.collectCandidates(0);
expect(candidates.tokens.size, "Test 2").to.equal(3);
expect(candidates.tokens.has(ExprLexer.VAR), "Test 3").to.equal(true);
expect(candidates.tokens.has(ExprLexer.LET), "Test 4").to.equal(true);
expect(candidates.tokens.has(ExprLexer.ID), "Test 5").to.equal(true);
expect(candidates.tokens.get(ExprLexer.VAR), "Test 6").to.eql([ExprLexer.ID, ExprLexer.EQUAL]);
expect(candidates.tokens.get(ExprLexer.LET), "Test 7").to.eql([ExprLexer.ID, ExprLexer.EQUAL]);
expect(candidates.tokens.get(ExprLexer.ID), "Test 8").to.eql([]);
Given the other errors I was seeing, I'm assuming there's something fundamentally different about the JavaScript parser/lexer than the ones generated by antlr4ts-cli
and something is still missing. Any help or insight as to what might be different would be useful.
from antlr4-c3.
I created a "pure" TypeScript project and used antlr4ts-cli
to create the parser and lexer classes, and I am not seeing this issue.
from antlr4-c3.
I am also seeing this same issue when using the antlr4
js package with a JavaScript
antlr target.
Using:
- antlr4 4.7.2
- anltr4-cs 1.1.11
My grammar test (based on the docs):
grammar Expr;
expression: assignment | simpleExpression;
assignment: (VAR | LET) ID EQUAL simpleExpression;
simpleExpression
: simpleExpression (PLUS | MINUS) simpleExpression
| simpleExpression (MULTIPLY | DIVIDE) simpleExpression
| variableRef
| functionRef
;
variableRef: ID;
functionRef: ID OPEN_PAR CLOSE_PAR;
VAR: [vV] [aA] [rR];
LET: [lL] [eE] [tT];
PLUS: '+';
MINUS: '-';
MULTIPLY: '*';
DIVIDE: '/';
EQUAL: '=';
OPEN_PAR: '(';
CLOSE_PAR: ')';
ID: [a-zA-Z] [a-zA-Z0-9_]*;
WS: [ \n\r\t] -> channel(HIDDEN);
The grammar is compiled using antlr4 -Dlanguage=JavaScript Expr.g4
per https://github.com/antlr/antlr4/blob/master/doc/javascript-target.md
My code (based on the docs):
import antlr4 from 'antlr4';
import * as c3 from 'antlr4-c3';
const suggestExpression = (input) => {
const inputStream = new antlr4.InputStream(input);
const lexer = new ExprLexer(inputStream);
const tokenStream = new antlr4.CommonTokenStream(lexer);
const parser = new ExprParser(tokenStream);
const errorListener = new antlr4.error.ErrorListener();
parser.addErrorListener(errorListener);
const tree = parser.expression();
const core = new c3.CodeCompletionCore(parser);
return core.collectCandidates(0);
};
The error:
CodeCompletionCore.js:67 Uncaught TypeError: Cannot read property 'index' of undefined
at CodeCompletionCore.collectCandidates (CodeCompletionCore.js:67)
The lines:
I've tried adding the inputStream to the parser via parser.inputStream
, but this results in an infinite loop in the while (true)
block just below. It seems that the tokenStream.LT(offset++)
call is returning -1
without a .type
so the loop just keeps going.
Is antlr4ts
required? Seems like using the antlr4
lib should work, too.
from antlr4-c3.
@kaidjohnson You know you are writing to a closed issue? Please don't hijack already finished issues but create a new one, if required.
Indeed there are differences between the JS and TS runtime versions. The authors of @antlr4ts insist on implementing their runtime in a way that is optimal for the TS language, thereby sacrificing compatibility. Best would be you open an issue against @antlr4ts, asking for more compatibility. However, I remember a discussion where people spoke about making the TS target the main one and generating the JS target from it. The current handwritten JS target would then be obsolete.
from antlr4-c3.
@mike-lischke It wasn't my intention to "hijack" anything. The issue I was seeing was the same as the user that posted the original issue here, and I thought I might get some additional insight into what the options were to get around it. Every git repo owner seems to have a different preference for creating tickets -- some complain about opening duplicates, some complain about extending closed tickets. It's always a 50/50 gamble; didn't mean to ruffle any feathers.
My current JS target was not handwritten, but rather generated via the core antlr4 library.
FWIW, I did work around the issue after a bit more trial-and-error by generating my parser and lexer via antlr4ts, converting the resulting ts (es2015 target) to js (using babel) -- the typescript es5 target wasn't handling the class conversion well -- and then using the core antlr4 runtime (not the ts version) for use in the browser. The parser needed to patched slightly to be compatible with the core Parser class from antlr4ts, but after that small patch (see below), I am getting tokens back.
Here's the patch, which essentially just copies over the missing method from antlr4ts:
import BaseExprParser from './ExprParser.ts';
class ExprParser extends BaseExprParser {
constructor(...args) {
super(...args);
// The antlr4ts Parser expects `tryLT` to be defined on the token stream
// and `LT` to defer to the `tryLT`.
// @see 'node_modules/antlr4ts/Parser.js
// @see 'node_modules/antlr4ts/BufferedTokenStream.js:203
this._input.tryLT = this._input.LT;
this._input.LT = function (k) {
const result = this.tryLT(k);
if (result === undefined) {
throw new RangeError('requested lookback index out of range');
}
return result;
};
}
}
For now, this resolves the issue for me and I am able to run the antlr4-c3 engine in the browser. Should I run into any other issues, I'll be sure to open a new ticket. Thanks!
from antlr4-c3.
Related Issues (20)
- Java port-> NPE happen when followSets contains FollowSetWithPath with null following member HOT 1
- symbolWithContextSync causing Maximum call stack size exceeded HOT 2
- Infinite loop in processRule()
- Port for Antlr4 runtime HOT 2
- Simple expression parser tests fail when executing out-of-order HOT 3
- About Typescript Runtime (antlr4s) HOT 3
- Candidate token's follow TokenList is inaccurate when the candidate's origin is ambiguous HOT 1
- nameless ScopedSymbol problem HOT 2
- [Question] Antlr-c3 with antlr4-ng runtime, issue with parser.getVocabulary() HOT 2
- How to use with plain JavaScript? HOT 2
- CodeCompleteCore constructor now references getVocabulary is this correct or should it be vocabulary HOT 2
- function to convert editor cursor position to token position HOT 1
- Symbol Tabel HOT 3
- About using esbuild to building HOT 3
- the library ist going to infinity loop HOT 1
- [Bug]: Unexpected behaviour in getAllSymbolsSync of ScopeSymbol
- Add command to esbuild HOT 2
- Autocompleting empty string causes error HOT 1
- Add .cjs output to package HOT 1
- Option to choose the parser rule for which we need the candidates HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from antlr4-c3.