This is valid syntax:

let foo = [,,];

1. see import
2. see export

The following link
https://camo.githubusercontent.com/1ea558cf5608a653c9e3b85bc9953181ee80d2d2/687474703a2f2f7465726869782e636f6d2f726174656c2d706572662d312e706e67

returns "Cannot proxy the given URL"

Currently we support:

• [...elems]
• f(...elems)

But we do not support {...props}.

While superfluous parens have no official semantics, they are actually used by browsers to perform laziness tricks:

var foo = function() { ... }; // Parsed lazily by the browser.
var bar = (function() { ... }); // Parsed eagerly by the browser.

It would be very useful to keep superfluous parens in such a case (or perhaps always). I don't know if that's what ratel does currently, so documenting the choice would be useful, too!

Perhaps it should be a parsing option?

A lot of the code is meant to be self explanatory, but some parts could use extra documentation, such as the different enum variants contained in grammar.rs.

Currently the tokenizer only parsers integers and floats. It needs to be expanded to allow for scientific notation, regex form: [eE][+-]?[0-9]+.

CC @cmtt

I've been thinking about coming with a name that isn't already taken on npm but also doesn't deviate from the badger-ness. One option I got, which sounds kinda cute and should be easy to remember is badgeroo.

Edit: we now have access to ratel on npm :).

The github organization can be then changed to:

ratel-rust / ratel-cli -> ratel on npm
ratel-rust / ratel-core -> ratel on crates.io

Sidenote: project logo could be a yellow JS-esque square with a head profile (plain black + white) of a honey badger in the bottom right corner.

I think one of the issues we might have right now that isn't super obvious is that it's actually hard to see anything working. Having to download, install and compile things just to play around with Ratel can be a lot to ask for, especially when our target is a JS crowd that's used to having online demos of everything.

That being said, because Ratel is insanely performant, putting it on a cheap EC2 with some very basic HTTP server to spit out compiled JS should be trivial. The server itself can be pure Rust and just pull ratel-core as a dependency. A simple S3 website with a try-it-out page that talks to the server should also be pretty easy to do.

Need to finally decide on a domain.

I'm currently working as a background task on a bridge between BinAST and Ratel. However, Ratel 0.7.0 and Ratel 0.8.0 seem to be very different beasts, with very different ASTs.

Which version should I target? Is Ratel 0.8.0 meant to be released soon?

Source maps are undoubtedly a necessity for development tools. We currently have some span helpers but don't have any code related to emitting sourcemaps yet.

• Attach 'src lifetime to all AST structs and enums.
• Replaces all instances of OwnedSlice throughout the code with &'src str.
• Remove source code from Program struct. Instead of trying to self-contain borrowing, which is difficult to impossible with the way borrowck works, we should rather embrace it and have the AST be an immutable borrow on the source. More in rationale below.
• Some unsafe code might be necessary for transmuting lifetimes when borrowck gets in the way, particularly when making slices from a source ref stored inside parser (borrowck can confuse parser lifetime with source lifetime).

Rationale

OwnedSlice is a footgun. It's efficient and rustc seems to optimize transforming it into &str at will without issues, however the method for creating OwnedSlice from non 'static slices introduces issues when it comes to explaining to end users of the AST when and why they should or shouldn't use it. Having some self-contained unsafe code in the parser is fine, having that unsafe code spread over virtually all parts of the code (transform and codegen) is not so great. Forcing users to use &'static str in transformer is, for the most part, a good limitation.

This should also allow us to separate the project into separate crates. While work is done on transformer, the parser and AST by themselves can be useful for other projects, e.g. should anyone want to build a JS linter with ratel-core.

Reproduce:

for (var i = 0 in {}) {}

To make sure that we write clean, idiomatic Rust, adding Clippy to the pipeline could be beneficial.

Rustfmt, unless it breaks something (aligning => in long match statements?) could be a nice addition as well.

https://cdnjs.cloudflare.com/ajax/libs/vue/2.5.17/vue.runtime.min.js

SyntaxError: Unexpected token at 6:83

> 6 | !function(t,e){"object"==typeof exports&&"undefined"!=typeof module?module.exports=e() // minified long line omitted...

https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Operators/Spread_operator

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/new.target

As follow-up task of #25, ratel-core should use rustdoc for documentation.

Generated content could be hosted using GitHub Pages or in scope of #26.

This is valid JavaScript:

let foo = { function() { } };
foo.function();

We need to allow keywords to extend ObjectKey as described here: #21 (comment)

Additionally Expression could use a KeywordMember variant where a keyword can be a property.

Lexicon needs to be changed in order to make keywords into it's own enum separate from tokens:

#[derive(Debug, PartialEq, Clone, Copy)]
pub enum KeywordKind {
    Break,
    Do,
    Case,
    Else,
    Catch,
    Export,
    Class,
    Extends,
    Return,
    While,
    Finally,
    Super,
    With,
    Continue,
    For,
    Switch,
    Yield,
    Debugger,
    Function,
    This,
    Default,
    If,
    Throw,
    Import,
    Try,
    Static,
}

#[derive(Debug, PartialEq, Clone, Copy)]
pub enum Token {
    EndOfProgram,
    Semicolon,
    Colon,
    Comma,
    ParenOpen,
    ParenClose,
    BracketOpen,
    BracketClose,
    BraceOpen,
    BraceClose,
    Keyword(KeywordKind), // replaces all above
    Operator(OperatorType),
    Declaration(VariableDeclarationKind),
    Reserved(ReservedKind),
    Identifier(OwnedSlice),
    Literal(Value),
    Template(TemplateKind),
}

Bonus 1: FatArrow should be a first class token, not a variant of OperatorType.
Bonus 2: OperatorType should be renamed to OperatorKind to keep the naming scheme consistent.

We need support for destructuring assignments, e.g.

const [ x, y ] = [ 1, 2 ];

If I read correctly the code, there is no universal node type.

This is the kind of thing that would be useful to write a converter between two AST:

fn ratel_to_binast(source: &ratel::Universal) -> Result<binast::Universal, ?> {
  ...
}

There may be alternatives, I'll think about that.

Parser needs to be able to handle parameters with default values.

ratel is a cool project. It seems that the rewrite branch is still quite active.
Do you have any roadmap or plan to integrate back to master branch?

As follow-up for #99, this issue is about adding optional support for comment nodes.

As @maciejhirsz suggested, comments should be gathered by the lexer as internal list.
This way, f.e. all comments of a function can be pulled before the declaration.

Please note that there are three types of valid comments in JavaScript:

• Line comments // foo
• Block comments /* bar */
• HTML comments <!-- baz --> (:angry:)

ECMAScript 2015: https://www.ecma-international.org/ecma-262/6.0/#sec-debugger-statement
ECMAScript 2017: https://www.ecma-international.org/ecma-262/8.0/#prod-DebuggerStatement

Currently looking at converting ratel -> binast. What's ratel::grammar::Statement::Transparent?

super() and super.* support, self explanatory.

Missing support for try & catch.

Allow for Multiplication token after Function keyword.

In order to have the REPL up-to-date with each master deployment, it should be deployed to a static web host during CI.

Additionally, wasm files should be served with application/wasm as MIME type. This ensures that browsers can compile the application while streaming the resource.

Currently, the following error is logged:

wasm streaming compile failed: TypeError: Failed to execute 'compile' on 'WebAssembly':
Incorrect response MIME type. Expected 'application/wasm'.
falling back to ArrayBuffer instantiation

I'm currently working on the JavaScript Binary AST TC39 proposal. Part of this proposal is an official AST for JavaScript. While this AST is not final yet, it is stabilizing, so I figured it might be of interest to you: WIP specs.

Extracting a Rust ADT from the WIP specs is pretty easy. If you need, I have code that does it already, and I figure I'll publish it as a separate crate soon.

x 
++ 
y

and

x 
-- 
y

Should be parsed as x; ++y; and x; --y; respectively.
They are currently parsed as x++; y; and x--; y;

JavaScript Syntax:

in operator

let obj = {"a": 1, "b": 2};
if ( "a" in obj === false ) throw new Error('Ooops ...');

Codgen:

var obj={"a":1,"b":2};if("a"inobj===!1)throw new Error('Ooops ...');

"a"inobj need space.

Apparently we are missing following tokens:

• &&= should be OperatorLogicalAndAssign*
• ||= should be OperatorLogicalOrAssign*
• # ? proposed use for private fields
• @ ? proposed use for decorators
• :: should be OperatorBind
• .. ? E4X specific, not sure if we even need it

Editing the enum will require altering all lookup tables, which is a bit of a chore.

* - those are apparently invalid in ECMAScript, some parsers recognize them internally as tokens but don't parse them.

Keywords:

• NaN / +NaN / -NaN --> std::f64::NAN
• Infinity / +Infinity --> std::f64::INFINITY
• -Infinity --> std::f64::NEG_INFINITY

These keywords need to be parsed into Literal, not an Identifier .

There are several other Rust projects (some launched recently) that deals with JavaScript sources. Let's see if we can get some ideas from their design.

https://github.com/nathan/pax by @nathan
https://github.com/swc-project/swc by @kdy1
https://github.com/FreeMasen/RESS by @FreeMasen

On this thread, let's focus on differences or ideas on:

• Parser structure
• Crate separation/modularization
• Project goals: beyond parsers, what should we focus on? Transpiling, minification/optimization, and bundling is what I'm aware of currently.

Library authors: are you interested in joining force with Ratel? Are there anything on library design that we should change? (Pardon this friendly ping, sorry if you get annoyed by a notification. Feel free to unsubscribe in any case.)

Is there a plan for supporting TypeScript syntax? I mean it just supports the syntax without doing actual type checking.

Hi, Have plan to support JSX syntax ?

Tokenizer needs to have unit tests to make sure that at least all operators are correctly turned into tokens.

Hi, the parser is not ready for parse unicode, right ?

The master branch:

• https://github.com/ratel-rust/ratel-core/blob/master/core/src/tokenizer.rs#L39-L57
• https://github.com/ratel-rust/ratel-core/blob/master/core/src/tokenizer.rs#L978-L980

The rewrite branch:

• https://github.com/ratel-rust/ratel-core/blob/rewrite/ratel/src/lexer/util.rs#L12-L30
• https://github.com/ratel-rust/ratel-core/blob/rewrite/ratel/src/lexer/mod.rs#L60-L78
• https://github.com/ratel-rust/ratel-core/blob/rewrite/ratel/src/lexer/labels.rs#L4-L31

JavaScript Unicode Name Example:

这是一个名称 = "世界 ( World )！";

console.log(这是一个名称.length);
console.log(`Hello, ${这是一个名称}`);

Need to add support for RegExp literals.

In parser if an expression begins with a Division operator, call a tokenizer method that reads the body and flags of a regular expression.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/do...while
http://www.ecma-international.org/ecma-262/5.1/#sec-12.6.1

For loops should allow body to be a single non-block statement.

Line: https://github.com/ratel-rust/ratel-core/blob/master/ratel/src/parser/expression.rs#L276-L277

parse f(1,2) is ok.
but parse f(1,) is error.

Reproduce:

const a = true ? console.log('foo') : null;

Template strings are actually quite complicated, since they can create 4 different grammar elements:

* `foobar`
* `foo${
* }bar{
* }baz`

Currently we only allow for class statements, but class expressions are also possible:

const foo = class { }

Just like with functions, class expression may have a name while a class statement must have a name. Other than that, parsing and transformation for class expressions and statements need to be DRY.

https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Functions/get
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Functions/set

Hey guys, I'm using this crate to build a control flow graph for JS code and I'm wondering how you would implement node parents. I'm reasonably new to Rust and I'm not sure where to start. I'm basing my work on the rewrite branch.