mck89 / peast Goto Github PK

• Logical Assignment Operators - Proposal - Commit
• Numeric separators - Proposal - Commit

Hi there,

it seems that the following code cannot be parsed in this configuration:

		$peast_options = [
			'sourceType' => Peast::SOURCE_TYPE_MODULE,
			'comments'   => true,
		];
		$ast           = Peast::latest( 'const HASH_RE = /#/g;', $peast_options )->parse();

When setting a new value to a StringLiteral using the setValue method all unicode escapes are escaped again in Utils::quoteLiteralString.

Example:

\u0076\u0061\u0072\u0069\u0061\u0062\u006c\u0065\u0073 turns into \\u0076\\u0061\\u0072\\u0069\\u0061\\u0062\\u006c\\u0065\\u0073

After running composer install --no-dev --prefer-dist --optimize-autoloader we noticed the following error:

Generating optimized autoload files

  In ClassMapGenerator.php line 73:

    Could not scan for classes inside "vendor/mck89/peast/test/Peast" which does not appear to be a file nor a folder

Is this related to the following:

The test folder is part of export-ignore but is registered in the composer.json autoload property.

Should it be moved the autoload-dev property?

    "autoload": {
        "psr-4": {
            "Peast\\": "lib/Peast/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "Peast\\test\\": "test/Peast/"
        }
    },

https://getcomposer.org/doc/04-schema.md#autoload-dev

Peast currently fails to parse this code:

$source = "var a=b?.05:-.05";
$ast = Peast\Peast::latest($source)->parse();

PHP Fatal error:  Uncaught Peast\Syntax\Exception: Unexpected: 05 in /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php:305
Stack trace:
#0 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3423): Peast\Syntax\ParserAbstract->error()
#1 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3269): Peast\Syntax\Parser->parseLeftHandSideExpression()
#2 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3220): Peast\Syntax\Parser->parsePostfixExpression()
#3 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3131): Peast\Syntax\Parser->parseUnaryExpression()
#4 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3093): Peast\Syntax\Parser->parseLogicalBinaryExpression()
#5 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3049): Peast\Syntax\Parser->parseConditionalExpression()
#6 /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2911): Peast\Syntax\Parser->parseAssignmentExpression()
#7 /private/tmp/peast in /private/tmp/peast/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php on line 305

{
  "type": "Program",
  "start": 0,
  "end": 16,
  "body": [
    {
      "type": "VariableDeclaration",
      "start": 0,
      "end": 16,
      "declarations": [
        {
          "type": "VariableDeclarator",
          "start": 4,
          "end": 16,
          "id": {
            "type": "Identifier",
            "start": 4,
            "end": 5,
            "name": "a"
          },
          "init": {
            "type": "ConditionalExpression",
            "start": 6,
            "end": 16,
            "test": {
              "type": "Identifier",
              "start": 6,
              "end": 7,
              "name": "b"
            },
            "consequent": {
              "type": "Literal",
              "start": 8,
              "end": 11,
              "value": 0.05,
              "raw": ".05"
            },
            "alternate": {
              "type": "UnaryExpression",
              "start": 12,
              "end": 16,
              "operator": "-",
              "prefix": true,
              "argument": {
                "type": "Literal",
                "start": 13,
                "end": 16,
                "value": 0.05,
                "raw": ".05"
              }
            }
          }
        }
      ],
      "kind": "var"
    }
  ],
  "sourceType": "module"
}

Same issue for {a:b?.05:-.05}. But there's no parse error when the dot is not followed by 0 as in var a=b?.5:-.05. var a=b?0.05:-.05 is working too.

Initially reported in wp-cli/i18n-command#297.

• Peast Version: 461fbe9
• Issue: When rendering MemberExpression with computed = false, the renderer won't check if property is a legal Identifier
• Reproduce:

1. Parse the following code

a[233] = false;
b['xx-dd'] = true;

2. Traverse the AST and execute $node->setComputed(false);
3. Render the script
4. The render result can be

a.233 = false;
b.xx-dd = true;

Which is illegal and will cause syntax error.

The following code cannot be parsed with Peast:

throw new TypeError("Hello\nWorld")

While this works:

throw new TypeError("Hello World")

Getting the following error:

Peast\Syntax\Exception: Unterminated string

AST for the above code:

{
  "type": "Program",
  "start": 0,
  "end": 35,
  "body": [
    {
      "type": "ThrowStatement",
      "start": 0,
      "end": 35,
      "argument": {
        "type": "NewExpression",
        "start": 6,
        "end": 35,
        "callee": {
          "type": "Identifier",
          "start": 10,
          "end": 19,
          "name": "TypeError"
        },
        "arguments": [
          {
            "type": "Literal",
            "start": 20,
            "end": 34,
            "value": "Hello\nWorld",
            "raw": "\"Hello\\nWorld\""
          }
        ]
      }
    }
  ],
  "sourceType": "module"
}

The following JS results in an error:

function test() {
    function set() {}

    return { set };
}

Using:

\Peast\Peast::latest($code, ['jsx' => true, 'sourceType' => \Peast\Peast::SOURCE_TYPE_MODULE ])->parse();

Error message with backtrace:

Peast\Syntax\Exception: Unexpected: }

lib/Peast/Syntax/ParserAbstract.php:305
lib/Peast/Syntax/Parser.php:2639
lib/Peast/Syntax/Parser.php:2816
lib/Peast/Syntax/Parser.php:2762
lib/Peast/Syntax/Parser.php:3618
lib/Peast/Syntax/Parser.php:3271
lib/Peast/Syntax/Parser.php:3199
lib/Peast/Syntax/Parser.php:3150
lib/Peast/Syntax/Parser.php:3077
lib/Peast/Syntax/Parser.php:3039
lib/Peast/Syntax/Parser.php:2995
lib/Peast/Syntax/ParserAbstract.php:349
lib/Peast/Syntax/Parser.php:2971
lib/Peast/Syntax/ParserAbstract.php:213
lib/Peast/Syntax/Parser.php:732
lib/Peast/Syntax/Parser.php:398
lib/Peast/Syntax/Parser.php:359
lib/Peast/Syntax/Parser.php:338
lib/Peast/Syntax/ParserAbstract.php:213
lib/Peast/Syntax/Parser.php:1633
lib/Peast/Syntax/ParserAbstract.php:213
lib/Peast/Syntax/Parser.php:1469
lib/Peast/Syntax/Parser.php:431
lib/Peast/Syntax/Parser.php:357
lib/Peast/Syntax/Parser.php:1982
lib/Peast/Syntax/Parser.php:491
lib/Peast/Syntax/Parser.php:197

AST for the JS snippet above:

{
  "type": "Program",
  "start": 0,
  "end": 62,
  "body": [
    {
      "type": "FunctionDeclaration",
      "start": 0,
      "end": 62,
      "id": {
        "type": "Identifier",
        "start": 9,
        "end": 13,
        "name": "test"
      },
      "expression": false,
      "generator": false,
      "async": false,
      "params": [],
      "body": {
        "type": "BlockStatement",
        "start": 16,
        "end": 62,
        "body": [
          {
            "type": "FunctionDeclaration",
            "start": 22,
            "end": 39,
            "id": {
              "type": "Identifier",
              "start": 31,
              "end": 34,
              "name": "set"
            },
            "expression": false,
            "generator": false,
            "async": false,
            "params": [],
            "body": {
              "type": "BlockStatement",
              "start": 37,
              "end": 39,
              "body": []
            }
          },
          {
            "type": "ReturnStatement",
            "start": 45,
            "end": 60,
            "argument": {
              "type": "ObjectExpression",
              "start": 52,
              "end": 59,
              "properties": [
                {
                  "type": "Property",
                  "start": 54,
                  "end": 57,
                  "method": false,
                  "shorthand": true,
                  "computed": false,
                  "key": {
                    "type": "Identifier",
                    "start": 54,
                    "end": 57,
                    "name": "set"
                  },
                  "kind": "init",
                  "value": {
                    "type": "Identifier",
                    "start": 54,
                    "end": 57,
                    "name": "set"
                  }
                }
              ]
            }
          }
        ]
      }
    }
  ],
  "sourceType": "module"
}

Program:

$src = '{ a: 1 }.a';
$ast = Peast::latest($src)->parse();

Result:

PHP Fatal error:  Uncaught Peast\Syntax\Exception: Unexpected: . in .../vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php:306
Stack trace:
#0 .../vendor/mck89/peast/lib/Peast/Syntax/Parser.php(201): Peast\Syntax\ParserAbstract->error()
#1 .../tests/test.php(34): Peast\Syntax\Parser->parse()
#2 {main}
  thrown in .../vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php on line 306

Expected result ($ast pseudo dump):

Array
(
    [type] => Program
    [body] => Array
        (
            [0] => Peast\Syntax\Node\ExpressionStatement Object
                (
                            [expression:protected] => Peast\Syntax\Node\MemberExpression Object
                                (
                                    [object:protected] => Peast\Syntax\Node\ObjectExpression Object
                                        (
                                            [properties:protected] => Array
                                                (
                                                    [0] => Peast\Syntax\Node\Property Object
                                                        (
                                                            [key:protected] => Peast\Syntax\Node\Identifier Object
                                                                (
                                                                    [name:protected] => a
                                                                )

                                                            [value:protected] => Peast\Syntax\Node\NumericLiteral Object
                                                                (
                                                                    [value:protected] => 1
                                                                )
                                                        )
                                                )
                                        )

                                    [property:protected] => Peast\Syntax\Node\Identifier Object
                                        (
                                            [name:protected] => a
                                        )
                                )
                )
        )
)

First of all, thank you for this package! It is helping a lot.

I was wondering if it's possible to query not just for values but for key names as well. Something like this:

$ast = Peast::latest($source)->parse();

$ast->query("Property[key='something']");

A return statement which returns a multi line template literal is parsed as a return statement without an argument and the template literal is parsed a single expression afterwards:

$source = "function templateLiteral() {
    return `
Hello World`;
}";
$tree = Peast\Peast::latest($source, [])->parse();

$renderer = new \Peast\Renderer;
$pp = $renderer->setFormatter(new \Peast\Formatter\PrettyPrint)->render($tree);

var_dump($pp);
//  function templateLiteral () {
//	return;
//	`
//  Hello World`;
//  }

In PHP 8.x is seems there's a parse error as the same invoking command works under PHP 7.4

PHP Parse error:  syntax error, unexpected fully qualified name "\Syntax\Node\Expression", expecting "," or ";" in phar:///opt/homebrew/Cellar/wp-cli/2.4.0/bin/wp/vendor/mck89/peast/lib/Peast/Syntax/Node/JSX/JSXOpeningElement.php on line 13

What a great library!
Just out of curiosity i wanted to know if someone has implemented a php transpiler.
The parser seems rock solid and i think it shouldnt be too hard to implement a subset ...

There seems to be an issue parsing async function declarations, couldn't find an existing or old issue about this.

Given:

Foo = {
  async bar() {},
  baz: async () => {},
}

Run through:

$ast = Peast::latest($data)->parse();
$renderer = new Renderer();
$renderer->setFormatter(new CompactFormatter());
print $renderer->render($ast);

You would expect:

Foo={async bar(){},baz:async()=>{}};

But instead you get:

Foo={bar(){},baz:async()=>{}};

Steps to reproduce aren't mine, just copying from: https://www.drupal.org/project/drupal/issues/3374660

The following code is removing the block statement when rendered, leading to a syntax error in the final output.

console.log('Start');

lbl: {
    console.log('Block Start');
    for (let i = 0; i < 5; i++) {
        console.log(`Loop 1 ${i}`);
    }

    for (let i = 0; i < 5; i++) {
        console.log(`Loop 2 ${i}`);
        if (i === 2) {
            break lbl;
        }
    }
}

console.log('End');

rendered to

console.log('Start');
lbl:
console.log('Block Start');
for (let i = 0; i < 5; i++) {
	console.log(`Loop 1 ${i}`);
}
for (let i = 0; i < 5; i++) {
	console.log(`Loop 2 ${i}`);
	if (i === 2) {
		break lbl;
	}
};
console.log('End');

Not really a bug, but I thought it would be worth to mention.

Runinng peast parser with php with enabled xdebug may lead to fatal error, as xdebug thinks we are inside infinite loop and exit the process forcefully.

It happens to me when I parse files with wp i18n command and my environment happens to have xdebug enabled by default.

Simply prefixing called command with env XDEBUG_MODE=off works around that issue.

Formatter/Compact does not introduce a new line character after the single line comment.
This causes the remaining instructions to be ignored and break the execution

this is an example of comment causing this issue

//# sourceMappingURL=

Hi,

The Drupal project is considering adding peast as a dependency to provide PHP-based shortening of JS files, by using the compact formatter to remove comments mainly. https://www.drupal.org/project/drupal/issues/3302755

Before that can be approved by the maintainers we have to perform a standard stability review.
If you could answer these questions it would be very helpful for our due diligence!

Do you have any official policies with regards to:

• Release windows/cadence
  • For example, do they happen as necessary on any given day, or on a set schedule after a certain passage of time (e.g. once a month)? From what we've seen there is no set release schedule, and there is a Patch releases every 2-4 months, minor releases every 1-2 years. Is that accurate?
• Backwards compatibility guarantees
  • Meaning, is there any guarantee that a given version will be supported for some period of time, like an LTS? (for the PHP API part)
  • I'm assuming the different JS versions are only going to be added, so that support for parsing ES2016 will not be dropped from the library.
• Security releases
  • For example, does more than one version receive security fixes, or only the current version?

I couldn't find the security policy on github so a couple additional questions:

1. If a vulnerability was found, do you have a process for privately reporting and fixing those issues?
2. Would you be willing to work with the Drupal Security Team to coordinate the timing of a new release if necessary?

I would really appreciate any info you can provide, and please let me know if anything is unclear.

Thanks!

Hi there,

I've come across a failure to parse what appears to be a valid JS file;
Source: https://api-platform.com/component---src-pages-index-js-7905c18163e9101ccbec.js
Command: Peast::latest($src, ['comments' => false, 'strictEncoding' => false, 'jsx' => true])->parse();

I've also come across a failure to tokenize a file that was parsed successfully;
Source: https://api-platform.com/framework-2d6b7d2e2be378715b78.js
Command: Peast::latest($node->render(new Compact))->tokenize();

Any pointers (assuming it is not a bug) would be much appreciated.

Electro

Exponentiation operator has right to left associativity.

Program:

$src = 'tmp = 2 ** 3 ** 4;';
$ast = Peast::latest($src)->parse();

$ast dump:

...
                            Peast\Syntax\Node\BinaryExpression Object
                                (
                                    [operator:protected] => **
                                    [left:protected] => Peast\Syntax\Node\BinaryExpression Object
                                        (
                                            [operator:protected] => **
                                            [left:protected] => Peast\Syntax\Node\NumericLiteral Object
                                                (
                                                    [value:protected] => 2
                                                )
                                            [right:protected] => Peast\Syntax\Node\NumericLiteral Object
                                                (
                                                    [value:protected] => 3
                                                )
                                        )
                                    [right:protected] => Peast\Syntax\Node\NumericLiteral Object
                                        (
                                            [value:protected] => 4
                                        )
                                )
...

Expected result:

...
                            Peast\Syntax\Node\BinaryExpression Object
                                (
                                    [operator:protected] => **
                                    [left:protected] => Peast\Syntax\Node\NumericLiteral Object
                                        (
                                            [value:protected] => 2
                                        )
                                    [right:protected] => Peast\Syntax\Node\BinaryExpression Object
                                        (
                                            [operator:protected] => **
                                            [left:protected] => Peast\Syntax\Node\NumericLiteral Object
                                                (
                                                    [value:protected] => 3
                                                )
                                            [right:protected] => Peast\Syntax\Node\NumericLiteral Object
                                                (
                                                    [value:protected] => 4
                                                )
                                        )
                                )
...

References:

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Exponentiation
• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence
• https://github.com/tc39/test262/blob/main/test/language/expressions/exponentiation/exp-operator-precedence-update-expression-semantics.js#L35
• https://astexplorer.net

By comparing the minified JS of jQuery and the output of PEAST, I noticed PEAST add emptyy parenthesis for new operator.

For example, in original jQuery minified file, you have:

return +new Date

WIth PEAST, it becomes:

return +new Date();

It seems these parenthesis are not needed when no arguments are passed to the constructor.

In file lib\Peast\Syntax\Utils.php, the method stringToUTF8Array can in some circumstances return false. Indeed, preg_split can return false. In such situation, your code output a lot of warnings and crashes randomly after.

It would be cool to add a check and throw an exception. You can get the error preg_split encountered using preg_last_error function.

The most common case is badly encoded UTF-8 file (preg_last_error returns 4); which is the situation I encountered myself : an UTF-8 file with an ISO-8859-1 characters in a comment....

    /**
     * Converts a string to an array of UTF-8 characters
     * 
     * @param string $str String to convert
     * 
     * @return array
     */
    static public function stringToUTF8Array($str)
    {
        return $str === "" ?
               array() :
               preg_split('//u', $str, null, PREG_SPLIT_NO_EMPTY);
    }

Originally reported at wp-cli/i18n-command#280, Peast fails to parse the following JSX using the <>/</> shorthand syntax for React fragments:

import React from 'react';

export default class Example extends React.Component {
  render() {
    return (
        <div className="example">
          <>
          </>
        </div>
    );
  }
}

Errror:

Peast\Syntax\Exception: Unexpected: >

Fragments are being parsed fine if they're on the outmost side though:

export default class Example extends React.Component {
  render() {
    return (
      <>
        <div className="example">
        </div>
      </>
    );
  }
}

The parser does not support dynamic import expressions (e.g. used by webpack)

import('some-file.js').then(_ => {})

Workaround: Before parsing the file, substitute all import\s+( strings with an alternative token, it will then be exposed as function call in the AST.

The following currently fails:

$code = '8e-5.toFixed(3)';

\Peast\Peast::latest($code, ['jsx' => true, 'sourceType' => \Peast\Peast::SOURCE_TYPE_MODULE ])->parse();

The issue seems to be the t of toFixed, thinking that it's still part of the notation.

Error:

Peast\Syntax\Exception: Invalid exponential notation

lib/Peast/Syntax/Scanner.php:655
lib/Peast/Syntax/Scanner.php:1485
lib/Peast/Syntax/Scanner.php:869
lib/Peast/Syntax/Scanner.php:764
lib/Peast/Syntax/Parser.php:2165
lib/Peast/Syntax/Parser.php:1978
lib/Peast/Syntax/Parser.php:491
lib/Peast/Syntax/Parser.php:197

AST for the above code:

{
  "type": "Program",
  "start": 0,
  "end": 15,
  "body": [
    {
      "type": "ExpressionStatement",
      "start": 0,
      "end": 15,
      "expression": {
        "type": "CallExpression",
        "start": 0,
        "end": 15,
        "callee": {
          "type": "MemberExpression",
          "start": 0,
          "end": 12,
          "object": {
            "type": "Literal",
            "start": 0,
            "end": 4,
            "value": 0.00008,
            "raw": "8e-5"
          },
          "property": {
            "type": "Identifier",
            "start": 5,
            "end": 12,
            "name": "toFixed"
          },
          "computed": false,
          "optional": false
        },
        "arguments": [
          {
            "type": "Literal",
            "start": 13,
            "end": 14,
            "value": 3,
            "raw": "3"
          }
        ],
        "optional": false
      }
    }
  ],
  "sourceType": "module"
}

Describe the current, buggy behavior

When running wp i18n make-pot . languages/shp_gantrisch_adb.pot on the command line in a site shell from the Local app on Mac OSX, a very long error message is generated. The initial message is Warning: Uncaught Error in exception handling during call to Error::__toString() in phar:///Applications/Local.app/Contents/Resources/extraResources/bin/wp-cli/wp-cli.phar/vendor/mck89/peast/lib/Peast/Syntax/Scanner.php on line 0

Describe how other contributors can replicate this bug
Run wp i18n make-pot . languages/shp_gantrisch_adb.pot on the command line in a site shell from the Local app on Mac OSX.

Describe what you would expect as the correct outcome

The POT file be generated without any error messages.

Let us know what environment you are running this on

OS:	Darwin 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64 x86_64
Shell:	/bin/zsh
PHP binary:	/Users/mark/Library/Application Support/Local/lightning-services/php-8.0.22+5/bin/darwin/bin/php
PHP version:	8.0.22
php.ini used:	/Users/mark/Library/Application Support/Local/run/_qwoyVNya/conf/php/php.ini
MySQL binary:	/Applications/Local.app/Contents/Resources/extraResources/lightning-services/mysql-8.0.16+6/bin/darwin/bin/mysql
MySQL version:	mysql  Ver 8.0.16 for macos10.14 on x86_64 (MySQL Community Server - GPL)
SQL modes:	ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
WP-CLI root dir:	phar://wp-cli.phar/vendor/wp-cli/wp-cli
WP-CLI vendor dir:	phar://wp-cli.phar/vendor
WP_CLI phar path:	/Users/mark/Projects/fse/app/public/wp-content/plugins/shp_gantrisch_adb
WP-CLI packages dir:	
WP-CLI global config:	/Applications/Local.app/Contents/Resources/extraResources/bin/wp-cli/config.yaml
WP-CLI project config:	
WP-CLI version:	2.6.0

Originally reported at wp-cli/i18n-command#280, it looks like Peast has troubles parsing comments within JSX in some cases.

The following file cannot be parsed, resulting in an Peast\Syntax\Exception: Unclosed { error:

import React from 'react';

export default class Example extends React.Component {
  render() {
    return (
        <div className="example">
          { /* This is a comment */ }
          { this.props.items.map( ( item ) => {
            return <div key={ item.id }>{ item.name }</div>;
          } ) }
        </div>
    );
  }
}

Hello, I have been using this library and it works like a charm, my only problem is that I can´t force the rendering of comments. I have been searching through the classes and found that the Base renderer has an attribute called $renderComments, but I don´t know how to use it or even if it can be used.

I have tried the following but with no luck :

Any help is appreciated
Duarte Guerreiro

Hello,
I have encountered some problems when using this code.
The arrow function does not add the return keyword after the conversion.

Here are the pre-conversion and post-conversion code.

JS:
Pre-conversion code :if(ships.some(s => s.ship.children('.boom').length == 0)){
console.log('111');
}

Post-conversion code:if (ships.some((s) => {
s.ship.children('.boom').length == 0;
//this line should "return s.ship.children('.boom').length == 0;"
})) {
console.log('111');
}

Peast is used in WP-CLI and so this was experienced when using Composer with WP-CLI.

I thought it might be helpful for the WP-CLI maintainers if I posted an issue here as well since it appears that two (2) of the four (4) notices come from Peast.

• For more details see this issue.

The following screenshot illustrates:

P.S. I hope you and your loved ones are weathering the COVID-19 storm successfully. The world is rooting for Italy.

Hi there,

This isn't a bug report but more of a request for more information.
I'm finding that for large JS files tokenization is taking 20 or more seconds in some cases.

I'm following the example in the docs, e.g;
$source = "large JS file";
$tokens = Peast\Peast::latest($source, $options)->tokenize();

I was wondering if there was any way I could increase the performance of the tokenize method (including possible code improvements and a pull request)?

Any help or pointers would be much appreciated
ElectroMW

An example of the JS that is taking a long time to tokenize;
https://pastebin.com/v7JALVV7
https://pastebin.com/RnEftpiT

Peast\Syntax\Exception with message "Unexpected #" throws then try to parse class code with private property like this one:

class ClassPrivateFld {
  #msg = 'hello world'
  get msg() {
    return this.#msg
  }
}

Hello,

It seems the compact mode breaks this script:

https://www.valeur-et-capital.com/wp-content/themes/cc_ad_valeur_et_capital_theme/elementor/widget_particules/js/particles.js?ver=5.2.2

The output is not working. Replacing with the original code made it working. I will investigate.

Is ES6 converted to ES5 support

As pointed on #23

Libraries like React are very popular and programs using it are mostly written using JSX as well. It would be useful to have it supported in this library.

My personal use case would be having a tool to extract gettext calls from a JSX source file.

I figure this could be done by introducing a new jsx option on the Parser class. By default it would be false.

Peast currently fails to parse this code:

const getTermsInfo = terms => ({
  terms,
  ...(terms === null || terms === void 0 ? void 0 : terms.reduce((accumulator, term) => { // This is line 28847
    const {
      mapById,
      mapByName,
      names
    } = accumulator;
    mapById[term.id] = term;
    mapByName[term.name] = term;
    names.push(term.name);
    return accumulator;
  }, {
    mapById: {},
    mapByName: {},
    names: []
  }))
});

I'm getting:

Peast\Syntax\Exception: Unexpected: (

Whereas this works:

const getTermsInfo = terms => {
    return {
        terms,
        ...(terms === null || terms === void 0 ? void 0 : terms.reduce((accumulator, term) => { // This is line 28847
            const {
            mapById,
            mapByName,
            names
            } = accumulator;
            mapById[term.id] = term;
            mapByName[term.name] = term;
            names.push(term.name);
            return accumulator;
        }, {
            mapById: {},
            mapByName: {},
            names: []
        }))
    };
}

This fails as well

const foo = () => ({ ...(void 0) }); // should return an empty object

This passes:

const foo = () => {
    return { ...(void 0) };
};

If the parsed code has regular expression literals containing quotes, these quotes are erroneously being parsed as opening or closing a string literal.

This will in turn cause the parser to throw an Unterminated string error once it hits the end of the line.

As an example, the WordPress Core code contains regular expression literals like this:

function escapeHTML(s) {
   var n = s;
   n = n.replace(/&/g, '&');
   n = n.replace(/</g, '&lt;');
   n = n.replace(/>/g, '&gt;');
   n = n.replace(/"/g, '&quot;');

   return n;
}

In this example, the regular expression literal /"/g will be parsed as opening a double-quoted string and throw an error once it hits the next line.

Related: wp-cli/i18n-command#98

The following JS code:

if (true) {
const CMS = 'Drupal';
}

parses to:
if (true) const CMS = 'Drupal';

Which result in: SyntaxError: Unexpected token 'const'

Peast\Syntax\Exception: Unexpected: else

            function bug() {
                for ( i = 0; i <= 1000; i++)
                    if (true) {
                        if (true)
                            do {
                                thing()
                            } while (true);
                        else 0 !== i ? thing() : i <= 10 ? thing() : thing();
                    }
            }

Hello,

Using PEAST on the minified version of jQuery v1.12.4 breaks the JS file.

Indeed, no errors occurs during the execution of the PHP code but the reconstructed JS code doesn't work correctly.

Code (where $js_content is the content of https://code.jquery.com/jquery-1.12.4.min.js) :

	//Generate the AST
	$ast = Peast\Peast::latest($js_content)->parse();
	//Create the renderer
	$renderer = new Peast\Renderer;
	//Associate the formatter
	$renderer->setFormatter(new Peast\Formatter\Compact);
	//Render the AST
	$js_content_new = $renderer->render($ast);

The resulting JS code differs from the original. I think it shouldn't: by this, I mean, each time I use PEAST on a minifed JS, this code outputs the same JS.

The problem doesn't appear if you just include the resulting JS file on a page. For me, the problem appeared when I included the minified JS of jQuery UI 1.11.4.

<html>
	<head>
		<script type="text/javascript" src="jquery_bad.js"></script>
		<script type="text/javascript" src="https://code.jquery.com/ui/1.11.4/jquery-ui.min.js"></script>
	</head>
	<body>
	</body>
</html>

The error you will get is coming from jQuery UI and is:

TypeError: a(...).outerWidth is not a function

Of course, no errors occurs if you use the original jQuery 1.12.4 JS minified file: https://code.jquery.com/jquery-1.12.4.min.js

• Export * as ns syntax (blocked by for estree/estree#202 to proceed)
• Optional chaining: https://github.com/tc39/proposal-optional-chaining (blocked by estree/estree#204)
• Nullish coalescing: https://github.com/tc39/proposal-nullish-coalescing (blocked by estree/estree#203 and estree/estree#204)

Hi, awesome project - I will be looking into it.

I would actually like to parse a toy programming language similar to JavaScript for fun; but for that I would need to extend your parser with a few new keywords.

What would be the proper approach if I wanted:

1. Typed variables; instead of "const", "let" and "var" I want "bool" etc.
2. New method modifier keywords (public, protected, private)

I don’t want to fork your parser; I want to extend it ideally. This toy language is a subset of JavaScript, and I would just validate the AST instead of modifying the parser too much.

The compact mode will transform:

"_"+ ++t+"__"

(notice the space after the first + sign)

by the invalid syntax:

"_"+++t+"__"

The compacter will output such code:

{
	'tab': 'active.tab',
	'sidepane',
}

when encoutering such JS code:

{
	'tab': 'active.tab',
	'sidepane': 'sidepane',
}

Unfortunately, this is not valid JS for latest Chrome:

Fails to parse the following JS:

(function(window, factory) {
	var lazySizes = factory(window, window.document, Date);
	window.lazySizes = lazySizes;
	//if(typeof module == 'object' && module.exports){
		module.exports = lazySizes;
	//}
})(typeof window != 'undefined' ?
      window : {}, function best() { function test() { var a = el[member].test[prop].method(arg).getBoundingClientRect(top); var item = typeof el; if (typeof item === 'undefined') { return true; } }; });

Would it be feasible to add TypeScript support?

Then it'd be easy for me convert typescript to javascript by walking the AST and omitting the typescript only features.

This could be used with a .htaccess rewrite to convert typescript to javascript on the fly, allowing for:

The http request to main.ts would return the transpiled javascript.

That script could cache the generated .js files. On a new request they would only be generated if the .ts file was newer than the generated .js file.

I think this could become a very popular and easy way to serve JavaScript from TypeScript on any web server with php, allowing developers to skip complicated node.js build configurations.

I'm trying to parse a string which contains this character escaped as UTF-16, but the result is incorrect. I debugged it a little bit and it seems that the issue is due to submitting the UTF-16 units to unicodeToUtf8 one-by-one, rather than decoding the unicode codepoint and then submitting that to unicodeToUtf8.

Here is a code that reproduces the problem:

// This doesn't work.
echo \Peast\Peast::latest('"\uD83D\uDE00"')->parse()->getBody()[0]->getExpression()->getValue() . "\n";
// These work.
echo json_decode('"\uD83D\uDE00"') . "\n";
echo \Peast\Syntax\Utils::unicodeToUtf8(0x0001F600) . "\n";
echo \Peast\Peast::latest('"\u{1F600}"')->parse()->getBody()[0]->getExpression()->getValue() . "\n";
echo \Peast\Peast::latest('"😀"')->parse()->getBody()[0]->getExpression()->getValue() . "\n";

// This also works (i.e. it shows the smiley face).
console.log("\uD83D\uDE00");

I tested this with PHP 7.4 and the latest master (b33fa0d).

hi

The licence suggests bsd-3 composer suggests MIT

could these by synced to the preferred licence

Parsing of bigint literals must be done inside parseNumericLiteral to allow bigint keys in objects.