Giter Club home page Giter Club logo

py-lua-parser's Introduction

py-lua-parser

https://travis-ci.org/boolangery/py-lua-parser.svg?branch=master

A Lua parser and AST builder written in Python.

It's both a development library and a command line tool.

Installation:

The package can be installed through pip:

$ python3.6 -m pip install luaparser

It will install the shell command 'luaparser'.

Options

These are the command-line flags:

Usage: luaparser [options] filename

CLI Options:
  --version                     Show program's version number and exit
  -h, --help                    Show this help message and exit
  -s, --source                  Source passed in a string
  -x, --xml                     Set output format to xml
  -o, --output                  Write output to file

Quickstart

Node structure

Each node contains the following data:

class Node:
        """Base class for AST node."""
        comments: Comments
        first_token: Optional[Token]
        last_token: Optional[Token]
        start_char: Optional[int]
        stop_char: Optional[int]
        line: Optional[int]

Working on AST tree

Minimal exemple:

from luaparser import ast

src = """
    local function sayHello()
      print('hello world !')
    end
    sayHello()
    """

tree = ast.parse(src)
print(ast.to_pretty_str(tree))

will display:

Chunk: {} 1 key
  body: {} 1 key
    Block: {} 1 key
      body: [] 2 items
        0: {} 1 key
          LocalFunction: {} 3 keys
            name: {} 1 key
              Name: {} 1 key
                id: "sayHello"
            args: [] 0 item
            body: [] 1 item
              0: {} 1 key
                Call: {} 2 keys
                  func: {} 1 key
                    Name: {} 1 key
                      id: "print"
                  args: [] 1 item
                    0: {} 1 key
                      String: {} 1 key
                        s: "hello world !"
        1: {} 1 key
          Call: {} 2 keys
            func: {} 1 key
              Name: {} 1 key
                id: "sayHello"
            args: [] 0 item

You can run through the list of all the nodes in the tree using ast.walk(tree):

from luaparser import ast
from luaparser import astnodes

tree = ast.parse("local foo = 'bar'")

for node in ast.walk(tree):
    if isinstance(node, astnodes.Name):
        process(node)

Alternatively, you can use a node visitor:

from luaparser import ast
from luaparser import astnodes

src = "local a = 42"

class NumberVisitor(ast.ASTVisitor):
    def visit_Number(self, node):
        print('Number value = ' + str(node.n))

tree = ast.parse(src)
NumberVisitor().visit(tree)

Rendering lua code

Warning

Experimental feature

exp = Chunk(Block([
    Forin(
        targets=[Name('k'), Name('v')],
        iter=[
            Invoke(
                source=Name('bar'),
                func=Name('foo'),
                args=[Number(42)]
            )
        ],
        body=Block([
            Call(func=Name('print'), args=[Name('k'), Name('v')])
        ]),

    )
]))

print(ast.to_lua_source(exp))

Will render:

for k, v in bar:foo(42) do
    print(k, v)
end

Command line

Given:

local function log(msg)
  print(msg)
end

log("hello world !")
$ luaparser source.lua

Will output:

{
    "Chunk": {
        "body": {
            "Block": {
                "body": [
                    {
                        "LocalFunction": {
                            "name": {
                                "Name": {
                                    "id": "log"
                                }
                            },
                            "args": [
                                {
                                    "Name": {
                                        "id": "msg"
                                    }
                                }
                            ],
                            "body": {
                                "Block": {
                                    "body": [
                                        {
                                            "Call": {
                                                "func": {
                                                    "Name": {
                                                        "id": "print"
                                                    }
                                                },
                                                "args": [
                                                    {
                                                        "Name": {
                                                            "id": "msg"
                                                        }
                                                    }
                                                ]
                                            }
                                        }
                                    ]
                                }
                            }
                        }
                    },
                    {
                        "Call": {
                            "func": {
                                "Name": {
                                    "id": "log"
                                }
                            },
                            "args": [
                                {
                                    "String": {
                                        "s": "hello world !"
                                    }
                                }
                            ]
                        }
                    }
                ]
            }
        }
    }
}

Command line

Documentation can be built with Sphinx:

$ cd doc
$ pip install -r requirements.txt
$ make html

py-lua-parser's People

Contributors

alkino avatar boolangery avatar mochaap avatar nanakipl avatar oltrep avatar penguinol avatar quarantin avatar ypaliy-vdoo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

py-lua-parser's Issues

Fail to reject unassigned global variable declaration

When a program contains an uninitialized declaration of a global variable. The parser should reject it, as per the specification. However the parser accepts and prints the AST, without raising an error.

Example:

var

Output:

{
    "Chunk": {
        "body": {
            "Block": {
                "body": [
                    {
                        "Name": {
                            "id": "var",
                            "start_char": 0,
                            "stop_char": 2,
                            "line": 1
                        }
                    }
                ],
                "start_char": 0,
                "stop_char": 2,
                "line": 1
            }
        },
        "start_char": 0,
        "stop_char": 2,
        "line": 1
    }
}

is ULengthOP Operator precedence Incorrected?

-- test.lua
local table3 = {}
table3[#table3 + 1] = 100

run
luaparser ./test.lua

output

{
    ...
    "ULengthOp": {
        "operand": {
            "AddOp": {
                "left": {
                    "Name": {
                        "id": "table3",
                        "start_char": 26,
                        "stop_char": 31,
                        "line": 2
                    }
                },
                "right": {
                    "Number": {
                        "n": 1,
                        "start_char": 35,
                        "stop_char": 35,
                        "line": 2
                    }
                },
            }
        }
    }
   ...
}

expected:

AddOp
{
    left: ULengthOp {}
    right: {...}
}

"expected chunk" but actually "didn't expect \\n or maybe <"

Hello,

I'm being stubborn and trying to do things outside my experience level, so I apologize if this is a dumb noob error. I haven't even tried to debug it properly yet but I did manage to find the smoking gun or maybe the smoke-in-mirrors; I see what went wrong but not why and have no idea how to investigate. I attached the source file and parsed tree for reference. Thank you for your work with this; I should be able to continue working through the file now in any case!

I went to try to "walk the tree" and went down a bit of a rabbit hole not realizing that "Chunk" was not actually a Python thing at all before digging into the error...


"E:\Python Projects\ScaleConstructor\venv2\Scripts\python.exe" "E:\Python Projects\ScaleConstructor\convert_from_lua.py" 
Traceback (most recent call last):
  File "E:\Python Projects\ScaleConstructor\convert_from_lua.py", line 11, in <module>
    tree = ast.parse(text)
           ^^^^^^^^^^^^^^^
  File "E:\Python Projects\ScaleConstructor\venv2\Lib\site-packages\luaparser\ast.py", line 14, in parse
    return Builder(source).process()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python Projects\ScaleConstructor\venv2\Lib\site-packages\luaparser\builder.py", line 231, in process
    raise SyntaxException("Expecting a chunk")
luaparser.builder.SyntaxException: Error: Expecting a chunk


chonker = open("rig","r")
text = str(chonker)
tree = ast.parse(text)
print(ast.to_pretty_str(tree))

my ultra basic debug output I stuck in places found this:

<luaparser.astnodes.Block object at 0x0000019DDA361DD0>
[@0,0:0='<',<35>,1:0]

the '<' looked out of place here, but wait, there's more...

"String": {
"s": "See the ShowDemoWindow() code in ReaImGui_Demo.lua. <- you are here!",
"delimiter": {},
"stop_char": 15553,
"line": 389
}

what I suspect could be the sneaky escape artist literally just a scroll away

"String": {
"s": "The \"Tools\" menu above gives access to: About Box, Style Editor,\\n",
"delimiter": {},
"start_char": 15243,
"stop_char": 15310,
"line": 384
}

escape_char_issue.zip

Rendering Some Lua Tables False

Example code:

local Table = {"a", "b", "c"};

I have used ast.to_lua_source for turning the ast into a lua code
The output i got:

local Table = {
    1 = "a",
    2 = "b",
    3 = "c",
}
;

What it should be:

local Table = {
    [1] = "a",
    [2] = "b",
    [3] = "c",
}
;

Used python code:

from luaparser import ast

src = """
local Table = {"a", "b", "c"};
"""

tree = ast.parse(src)
print(ast.to_lua_source(tree))

Verify assign visitor behavior

Hi,

First of all, my intention is to get the global variables in a source code.

For that, I created a test:

src = """
	local function sayOla(s)
		s.msg = "Ola!"
		print(s.msg)
	end
	
	local ola = {}
	sayOla(ola)
	"""
tree = ast.parse(src)
print(ast.to_xml_str(tree))

However, the assign in the 's' parameter variable seems like normal assign even with the variable 'ola' being local. It can be seen in xml content below.

I think that can be very hard to check if the variable pass through the function is local or not. But, it would be interesting to differentiate the assing of a function parameter or create a GlobalAssign. I don't know what would be better.

When I use the function visitor in CellularSpace.lua, it returns just the unique global function in that file, but if I use the assign visitor, it returns a lot of variables that are passed through the functions. You can check that.
CellularSpace.txt

<?xml version="1.0" ?>
<doc>
   <Chunk>
      <comments/>
      <body>
         <Block>
            <comments/>
            <body>
               <LocalFunction>
                  <comments/>
                  <name>
                     <Name>
                        <comments/>
                        <id>sayOla</id>
                     </Name>
                  </name>
                  <args>
                     <Name>
                        <comments/>
                        <id>s</id>
                     </Name>
                  </args>
                  <body>
                     <Block>
                        <comments/>
                        <body>
                           <Assign>
                              <comments/>
                              <targets>
                                 <Index>
                                    <comments/>
                                    <idx>
                                       <String>
                                          <comments/>
                                          <s>msg</s>
                                       </String>
                                    </idx>
                                    <value>
                                       <Name>
                                          <comments/>
                                          <id>s</id>
                                       </Name>
                                    </value>
                                 </Index>
                              </targets>
                              <values>
                                 <String>
                                    <comments/>
                                    <s>Ola!</s>
                                 </String>
                              </values>
                           </Assign>
                           <Call>
                              <comments/>
                              <func>
                                 <Name>
                                    <comments/>
                                    <id>print</id>
                                 </Name>
                              </func>
                              <args>
                                 <Index>
                                    <comments/>
                                    <idx>
                                       <String>
                                          <comments/>
                                          <s>msg</s>
                                       </String>
                                    </idx>
                                    <value>
                                       <Name>
                                          <comments/>
                                          <id>s</id>
                                       </Name>
                                    </value>
                                 </Index>
                              </args>
                           </Call>
                        </body>
                     </Block>
                  </body>
               </LocalFunction>
               <LocalAssign>
                  <comments/>
                  <targets>
                     <Name>
                        <comments/>
                        <id>ola</id>
                     </Name>
                  </targets>
                  <values>
                     <Table>
                        <comments/>
                        <fields/>
                     </Table>
                  </values>
               </LocalAssign>
               <Call>
                  <comments/>
                  <func>
                     <Name>
                        <comments/>
                        <id>sayOla</id>
                     </Name>
                  </func>
                  <args>
                     <Name>
                        <comments/>
                        <id>ola</id>
                     </Name>
                  </args>
               </Call>
            </body>
         </Block>
      </body>
   </Chunk>
</doc>

Unable to tell apart true indexing vs. syntactic sugar indexing

First, thanks for making this! I've been playing with it a bit to poke at ASTs and like that Lua is so simple to reason about!

Before getting into the bug, I want to put the code I was looking at in Lua:

> a = 'b'
> x = {a = 'a', b = 'b'}
> -- case 1
> print(x[a])
b
> -- case 2
> print(x['a'])
a
> -- case 3, equivalent to case 2, but using syntactic sugar
> print(x.a)
a

it seems that luaparser treats cases 1 & 3 as equivalent, instead of 2 & 3:

>>> from luaparser import ast
>>> case_1 = ast.parse("x[a]")
>>> case_2 = ast.parse("x['a']")
>>> case_3 = ast.parse("x.a")
>>> case_1 == case_3
True
>>> case_2 == case_3
False

If we look at the AST that's generated it seems it seems that for case 3 the idx field of the Index is erroneously a Name instead of a String:

>>> print(ast.toPrettyStr(case_3))

Chunk: {} 2 keys
  body: {} 2 keys
    Block: {} 2 keys
      body: [] 1 item
        0: {} 1 key          
          Index: {} 3 keys
            idx: {} 2 keys
              Name: {} 2 keys
                id: 'a'
            value: {} 2 keys
              Name: {} 2 keys
                id: 'x'

>>> print(ast.toPrettyStr(case_2))

Chunk: {} 2 keys
  body: {} 2 keys
    Block: {} 2 keys
      body: [] 1 item
        0: {} 1 key          
          Index: {} 3 keys
            idx: {} 2 keys
              String: {} 2 keys
                s: 'a'
            value: {} 2 keys
              Name: {} 2 keys
                id: 'x'

Ambiguous syntax detected

Hi

I've failed to reproduce the syntax error described in Lua runtime for both simple cases e.g.

io.write
('A')

(IDE One result)

and the example mentioned in the link in this comment:

z = (function(...) print(...); return "Bar"; end)
("Foo"):sub(1,1)

io.write(z)

(IDE One result)

They work fine in both the online Lua playground and IDE One, but fail to parse with this library getting this error:

File "/home/.../site-packages/luaparser/builder.py", line 697, in parse_tail
    raise SyntaxException(
luaparser.builder.SyntaxException: (2,4): Error: Ambiguous syntax detected

Was this a limitation of older Lua versions or ...?

For context my use case was static analysis of an existing working code base, and I was forced to patch it to be able to get it parsed by this library which was not ideal.

Is there any chance this error can be removed with ambiguity solved the same way Lua itself does? or was there another reason for this to be added that I missed? I didn't find any other issues/pull requests referencing this error.

Thanks.

Edit: for the test case used Lua runtime resolves it to as if there was no newline present
image

Strings are not being parsed

when I parse the string `'\0\n\ta', the escape sequences are not being parsed.

tree = ast.parse("a='\\0\\n\\ta'")

leads to the string node:

String: {} 3 keys
  s: '\\0\\n\\ta'
  delimiter: SINGLE_QUOTE

however, it should look like:

String: {} 3 keys
  s: '\x00\n\ta'
  delimiter: SINGLE_QUOTE

luaparser.utils.visitor.VisitorException: No visitor found for class <enum 'StringDelimiter'>

Using the basic command:

from luaparser import ast

src = """
local function sayHello()
print('hello world !')
end
sayHello()
"""

tree = ast.parse(src)
print(ast.to_pretty_str(tree))

I get this:
print(ast.to_pretty_str(tree))
File "~Python37\lib\site-packages\luaparser\ast.py", line 38, in to_pretty_str
return printers.PythonStyleVisitor(indent).visit(root)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 107, in visit
res += self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 107, in visit
res += self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 80, in visit
res += self.indent_str(False) + self.visit(itemValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 107, in visit
res += self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 107, in visit
res += self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 80, in visit
res += self.indent_str(False) + self.visit(itemValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 107, in visit
res += self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 80, in visit
res += self.indent_str(False) + self.visit(itemValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 37, in _visitor_impl
return method(self, arg)
File "~Python37\lib\site-packages\luaparser\printers.py", line 111, in visit
res += self.indent_str() + attr + ': ' + self.visit(attrValue)
File "~Python37\lib\site-packages\luaparser\utils\visitor.py", line 40, in _visitor_impl
raise VisitorException('No visitor found for class ' + str(type(arg)))
luaparser.utils.visitor.VisitorException: No visitor found for class <enum 'StringDelimiter'>

I am using python 3.7.9 and vs code.
Note that ~ is used to remove references to my local machine.

Failure to reject incorrect inputs: name token

Hi,

The parser doesn't reject some of the incorrect and incomplete Lua inputs.

As a short example, consider the following Lua string:

var_name

The parser parses and labels the input as Name, without raising any error!

In comparison to the official Lua grammar, and official Lua implementation, the input is invalid. This is because Name is not a statement. So, it can only be part of a statement, and is only allowed in certain contexts.

So my question is : is this behaviour intended? Or is it a parsing error?

Thank you.

Broken on 3.8, same error as issue #11

I know Python 3.8 isn't listed in the supported versions, but the Hello World example in the README is crashing with the same error as issue #11 and I'm wondering if the cause might be something similar.

Environment:

  • Python 3.8.7
  • OS: MacOS 11.2.3
  • Packages:
    • luaparser==3.0.0
    • antlr4-python3-runtime==4.7.2

ast.to_lua_source doesn't add parenthesis to expressions

Dependless of the input expression, ast.to_lua_source doesn't add any parenthesis.
Example:

>>> ast.to_lua_source(ast.parse("a=(1*2)+3"))
'a = 1 * 2 + 3'
>>> ast.to_lua_source(ast.parse("a=1*(2+3)"))
'a = 1 * 2 + 3'

While it is possible for the first expression to be written without parenthesis, the second one needs a pair in order to preserve the meaning.

Failure to parse chained comparisons

Hi,

Here is a Lua example:

if false == false == false then 
x = 2
end

When parsing this program, the parser returns the following:

error: Error: Expecting one of 'and', 'function', 'nil', 'not', 'or', 'then', 'true', '+', '-', '*', '/', '//', '%', '^', '#', '&', '|', '~', '>>', '<<', '(', '{', '...', '..', NAME, NUMBER, STRING at line 1, column 21

Note that the Lua compiler executes the program successfully.

Failure to reject incorrect inputs: missing function call arguments

Hi,

The parser doesn't reject incorrect function calls.

As a short example, consider the following Lua input:

var = function(...)
    (4)
end

The parser parses and labels function call (4) as a valid statement, without raising an error!

Here is the output:

{
    "Chunk": {
        "body": {
            "Block": {
                "body": [
                    {
                        "Assign": {
                            "targets": [
                                {
                                    "Name": {
                                        "id": "var",
                                        "start_char": 0,
                                        "stop_char": 2,
                                        "line": 1
                                    }
                                }
                            ],
                            "values": [
                                {
                                    "AnonymousFunction": {
                                        "args": [
                                            {
                                                "Varargs": {
                                                    "start_char": null,
                                                    "stop_char": null,
                                                    "line": null
                                                }
                                            }
                                        ],
                                        "body": {
                                            "Block": {
                                                "body": [
                                                    {
                                                        "Number": {
                                                            "n": 4,
                                                            "start_char": 25,
                                                            "stop_char": 25,
                                                            "line": 2
                                                        }
                                                    }
                                                ],
                                                "start_char": 24,
                                                "stop_char": 30,
                                                "line": 2
                                            }
                                        },
                                        "start_char": 6,
                                        "stop_char": 30,
                                        "line": 1
                                    }
                                }
                            ],
                            "start_char": 0,
                            "stop_char": 30,
                            "line": 1
                        }
                    }
                ],
                "start_char": 0,
                "stop_char": 30,
                "line": 1
            }
        },
        "start_char": 0,
        "stop_char": 30,
        "line": 1
    }
}

In comparison to the official Lua grammar, and official Lua implementation, the input is invalid. This is because a function call should be followed by one or more arguments.

So my question is : is this behaviour intended? Or is it a parsing error?

Thank you.

Fail to reject an invalid escape sequence

The parser fails to reject the following invalid input:

return "\u{XYZ}"

Only hexadecimal digits should be allowed in the escape sequence, which is not the case in the example above.

From the Lua specification, we have:

The UTF-8 encoding of a Unicode character can be inserted in a literal string with the escape sequence \u{XXX} (note the mandatory enclosing brackets), where XXX is a sequence of one or more hexadecimal digits representing the character code point.

Can you provide feedback on this issue please.

ULengthOp not correctly parsed

function definition:

function test_olength()
setting_name = "12345678"
if #setting >10 and setting_name == "user" then return 100 end
end

{
"test_olength": {
"Chunk": {
"body": {
"Block": {
"body": [
{
"Function": {
"name": {
"Name": {
"id": "test_olength",
"line": 1
}
},
"args": [
{
"Name": {
"id": "setting_name",
"line": 1
}
}
],
"body": {
"Block": {
"body": [
{
"Assign": {
"targets": [
{
"Name": {
"id": "setting_name",
"line": 2
}
}
],
"values": [
{
"String": {
"s": "12345678",
"delimiter": {},
"line": 2
}
}
],
"line": 2
}
},
{
"If": {
"test": {
"ULengthOp": {
"operand": {
"LAndOp": {
"left": {
"RGtOp": {
"left": {
"Name": {
"id": "setting",
"line": 3
}
},
"right": {
"Number": {
"n": 10,
"line": 3
}
},
"line": null
}
},
"right": {
"REqOp": {
"left": {
"Name": {
"id": "setting_name",
"line": 3
}
},
"right": {
"String": {
"s": "user",
"delimiter": {},
"line": 3
}
},
"line": null
}
},
"line": null
}
},
"line": 3
}
},
"body": {
"Block": {
"body": [
{
"Return": {
"values": [
{
"Number": {
"n": 100,
"line": 3
}
}
],
"line": 3
}
}
],
"line": 3
}
},
"line": null
}
}
],
"line": 2
}
},
"line": 1
}
}
],
"line": 1
}
},
"line": 1
}
}
}

as you see, the ULengthOp is the 1st depth operator which is wrong, it should be "LAndOp"

No visitor found for class astnodes.Comment

Hello,

Thanks for this good job.

I am assessing this lua parser to a project.

When uses the toXmlStr() method and there is a comment in the source, it raises an error:

from luaparser import ast

src = """
    -- comment
    local function sayHello()
      print('hello world !')
    end
    sayHello()
    """

tree = ast.parse(src)
print(ast.toXmlStr(tree))
File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 36, in _visitor_impl
    raise VisitorException('No visitor found for class ' + str(type(arg)))
luaparser.utils.visitor.VisitorException: No visitor found for class <class 'luaparser.astnodes.Comment'>

Parsing removes parentheses

Parsing Lua code seems to remove parentheses from the AST tree, causing problem when the code is rendered.
For example:
Parsing (1+5)/2 results in 1+5/2 (Causing unexpected math result)
Parsing (function() return "AnonymousFunction" end)() results in function() "AnonymousFunction" end() (Causing error in rendered Lua code as Anonymous Functions require closed parentheses around them)

Missing parentheses are a significant issue in math and programming and should be fixed as soon as possible,
Thank you.

Comments with Chinese characters are discarded

Dear maintainers,

When using the parser, I find that sometimes comments with Chinese characters are silently discarded.
Below is Lua code which can be used to reproduce the error:

   function setupRichText()
        richText.fitArea = false	-- 是否根据内容自适应高度
        richText.fitPerHeight = nil	-- 自适应的单行高度
        return richText
    end

After parsing the code above using ast.parse(), the 2nd comment ('-- 自适应的单行高度') cannot be found anywhere in the returned AST. However the 1st comment can be found in the AST as expected.

Would you please check if there is something wrong when processing comments in Chinese?

Add Docs website / build instructions

Is there any documentation readily available?
I have seen the docs folder, but there are no instructions on how to convert it into docs.
Please add either one so I know what I'm working with.

Output as a lua file?

Hallo,

Thanks for this wonderful project.
While I can happily parse a Lua file to an ast, if there a way to store a modified AST to a lua file?
Sorry, if this was already done before. Could you please help me to figure a way to do this if possible?

Thanks

Literal string gets evaluated as Lua code

Hi,

When a Lua string starts with prefix \u, the string is parsed as Lua code. For example:

a="\u9"

Returns the following output:

line 1:2 token recognition error at: '"\u'
line 1:6 token recognition error at: '"\n'
{
    "Chunk": {
        "body": {
            "Block": {
                "body": [
                    {
                        "Assign": {
                            "targets": [
                                {
                                    "Name": {
                                        "id": "a",
                                        "start_char": 0,
                                        "stop_char": 0,
                                        "line": 1
                                    }
                                }
                            ],
                            "values": [
                                {
                                    "Number": {
                                        "n": 9,
                                        "start_char": 5,
                                        "stop_char": 5,
                                        "line": 1
                                    }
                                }
                            ],
                            "start_char": 0,
                            "stop_char": 5,
                            "line": 1
                        }
                    }
                ],
                "start_char": 0,
                "stop_char": 5,
                "line": 1
            }
        },
        "start_char": 0,
        "stop_char": 5,
        "line": 1
    }
}

Digit 9 is evaluated to Number. I was expecting everything enclosed in quotes to be String.

Is that correct?

Wrong intent when calling to_lua_source()

for example, a lua file like:

local a = 0
if a == 0 then
    if a == 1 then
        if a == 2 then
            if a == 3 then

            end
        end
    end
end

I loaded the lua source by ast, and then rewrite the ast to source.

lua_ast = ast.parse(content)
tree = ast.parse(content)
print(ast.to_lua_source(tree))

the output is like

local a = 0
if a == 0 then
  if a == 1 then
      if a == 2 then
            if a == 3 then

            else

            end
      else

      end
  else

  end
else

end

to_lua_source issue when returning empty value

When you try to parse this simple function:

function test()
	return
end

And then call to_lua_source on the AST you get the following result:

function test()
	return False
end

We can see the resulting code is returning False which is a value from python.

No visitor found for class <class 'NoneType'>

When I am trying to use ast.walk() in my source project there is something that the parse is not recognizing.

The file source code which I am testing (rename it to '.lua):
CellularSpace.txt

Source:

src = open('CellularSpace.lua').read()
tree = ast.parse(src)
for node in ast.walk(tree):
	print(node)

The error:

    for node in ast.walk(tree):
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 24, in walk
    visitor.visit(root)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 130, in visit
    self.visit(node.body)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 135, in visit
    self.visit(node.body)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 125, in visit
    self.visit(n)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 226, in visit
    self.visit(node.body)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 135, in visit
    self.visit(node.body)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 125, in visit
    self.visit(n)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 25, in _visitor_impl
    return method(self, arg)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/ast.py", line 159, in visit
    self.visit(node.orelse)
  File "/usr/local/lib/python3.6/dist-packages/luaparser/utils/visitor.py", line 36, in _visitor_impl
    raise VisitorException('No visitor found for class ' + str(type(arg)))
luaparser.utils.visitor.VisitorException: No visitor found for class <class 'NoneType'>

[Bug] Some Comment Data Ignored

Hi,

It seems that although the parser can detect comments properly, it fails to assign all of it to the node (sometimes?). I know that sounds rather vague, but let me give you an example.

I have in a Lua file, the following comment at the top of the file:

--[[
- @file server.lua
- @brief This is the server file that the client connects to
--]]

When I dump the node data for this file, I can see that the entirety of that file's Chunk contents sees the whole comment:

Chunk: {} 4 keys
  comments: [] 1 item
    0: '--[[\n- @file server.lua\n- @brief This is the server file that the client connects to\n--]]'
...

However, if I use a Visitor class to handle finding the first item I want, @file ... it fails to find it. If I were to print the comment, since it's multiline, I get this:

====================
server.lua
--[[
- @brief Initialize the server protocol.
--]]
...
====================

To be certain, I also tried checking if the parser somehow put it as non-multiline:

====================
server.lua

====================

It should be noted, though, all the other comments are fine and intact, following a similar syntax.

That aside, thank you for creating this useful python package! :)

If you need to see my source code at all, please let me know and I'll gladly provide it. Currently I haven't pushed the python code, but the Lua code is readily available on my GitHub.


P.S. I also would like to know if there's a way to determine where the comments/functions/etc are, but I think that would require a feature request to add line numbers where each Node item starts.

Can't parse call expressions in LocalAssign

Given the following code:

local d, p, one, floor, mul, idx;
if d * d > p then
    local lo = floor(one * (2^-56 / 3) * mul / p)
end

Running luaparser:
py -m luaparser <path>

Expected result:
<successfully parsed chunk>

Error:
Expecting one of 'break', 'do', 'else', 'elseif', 'end', 'function', '::', 'return', ';' at line 3, column 21

Additional Info: builder.py

Seem like parse_expr_list inside parse_local has no actual way of handling functions call expression, and the floor call is resolved to a Name expression

have python2 version?

my python version is 2.x ,even i install success,but i can not import the module,can provide version for python2.x

Comments in the end of Node will be skipped

Examples:

Chunk Tail

local x = 1
-- comments will never be visited

or Function Tail

function ok()
  --print('comments will never be visited')
end

or Just Comments

-- comments will never be visited

Bitwise precedence is wrong

Looking at parser/Lua.g4, it seems that the assumed bitwise priority is above multiplication, while it is in fact below concat reference on operator precedence.
This is also the case in a small example.
In a lua 5.4 console:

> 1&2+3
1
> (1&2)+3
3

The parsed tree for "1&2+3":

AddOp: {} 3 keys
  left: {} 3 keys
    BAndOp: {} 3 keys
      left: {} 2 keys
        Number: {} 2 keys
          n: 1
      right: {} 2 keys
        Number: {} 2 keys
          n: 2
  right: {} 2 keys
    Number: {} 2 keys
      n: 3

Refusing to parse chained conditions without parentheses, eg "if a == b == c then"

If you try to parse

if a == b == c then end

it raises a syntax error

luaparser.builder.SyntaxException: Error: Expecting one of 'and', 'not', 'or', 'then', '+', '-', '*', '/', '//', '%', '^', '#', '&', '|', '~', '>>', '<<', '(', '{', '[', ':', '..', '.', STRING at line 1, column 13

However if you parse the equivelent

if (a == b) == c then end

then it works just fine.

found this bug whilst trying to parse this

if aJ == aK == (aI.A ~= 0) then 

luaparser.utils.visitor.VisitorException: No visitor found for class <class 'dict'>

Traceback (most recent call last):
File "c:\Users\hjaco\Downloads\lua obf\v2\main.py", line 79, in
code = ast.to_lua_source(tree)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\ast.py", line 40, in to_lua_source
return printers.LuaOutputVisitor(indent_size=indent).visit(root)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\printers.py", line 245, in visit
return self.visit(node.body)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\printers.py", line 251, in visit
"\n".join([self.visit(n) for n in node.body]), " " * (self._indent_size if self._level > 1 else 0)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\printers.py", line 251, in
"\n".join([self.visit(n) for n in node.body]), " " * (self._indent_size if self._level > 1 else 0)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\printers.py", line 382, in visit
+ self.visit(node.source)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\utils\visitor.py", line 29, in _visitor_impl
return method(self, arg)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\printers.py", line 550, in visit
return self.visit(node.id)
File "C:\Users\hjaco\AppData\Local\Programs\Python\Python310\lib\site-packages\luaparser\utils\visitor.py", line 40, in _visitor_impl
raise VisitorException("No visitor found for class " + str(type(arg)))
luaparser.utils.visitor.VisitorException: No visitor found for class <class 'dict'>

Input code:

local t1 = {}

function t1:sayHello()
  print("Hello")
end

t1:sayHello()

Python code:

stack = []
for node in ast.walk(tree):
    if isinstance(node, nodeType.Name):
        #node.id
        result = inStack(stack, node.id, False)
        if result[0]:
            node.id = stack[result[1]]
        else:
            if isinstance(prevNode, nodeType.LocalAssign) or isinstance(prevNode, nodeType.LocalFunction) or isinstance(node, nodeType.Function):
                name = genName(stack)
                stack.append({"old": node.id, "new": name})
                node.id = name
                print(node.id)
    prevNode = node

# convert to code
code = ast.to_lua_source(tree)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.