grammatech / sel Goto Github PK
View Code? Open in Web Editor NEWProgrammatic modification and evaluation of software
Home Page: https://grammatech.github.io/sel/#Top
License: Other
Programmatic modification and evaluation of software
Home Page: https://grammatech.github.io/sel/#Top
License: Other
Building with SBCL 2.1.10.195-bca540cd6 / ASDF 3.3.5 for quicklisp dist creation.
Trying to build commit id e5b4a22
software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
software-evolution-library/run-dump-store fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10418EC4A3}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
software-evolution-library/run-rest-server fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
software-evolution-library/run-test-parse fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
software-evolution-library/run-tree-sitter-interface fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:
; caught ERROR:
; during macroexpansion of (MATCH2*+ (JSON) (T) ...). Use *BREAK-ON-SIGNALS* to intercept.
; Symbol JSON-DOCUMENT is unbound in namespace PATTERN
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001BFC103}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/javascript-project" "file-type">
SEL used to work fine but recently I cannot load it and I get this error: 'System "gt/full" not found'
This can be found in the quicklisp file: "https://git.grammatech.com/synthesis/gt.git" which is a repo which is not accessible [publicly].
Hi, I'm having some trouble trying to run the examples.
Specifically, when I try to load the 003-neutral.lisp
example, the lisp interpreter throws No package named "SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER
.
So I manually load (ql:quickload :software-evolution-library/software/tree-sitter)
, but the interpreter says
> Error: The value :LOCAL-NICKNAMES is not of the expected type (MEMBER :USE :MIX :REEXPORT :USE-REEXPORT :MIX-REEXPORT :IMPORT-FROM :SHADOWING-IMPORT-FROM :NICKNAMES :DOCUMENTATION :SHADOW :EXPORT :INTERN :UNINTERN :RECYCLE).
> While executing: ASDF/PACKAGE-INFERRED-SYSTEM::PACKAGE-DEPENDENCIES, in process listener(1).
Also, the same error occurs when loading "(ql:quickload:software-evolution-library/software/clang)".
I've been haunted for a long time, so I decided to ask you for help.
Thanks.
Bo.
I get this:
;
; caught WARNING:
; Constant NIL conflicts with its asserted type (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING FUNCTION).
Full context and log is here: http://report.quicklisp.org/2019-07-23/failure-report/sel.html#software-evolution-library_test
Log is here: http://report.quicklisp.org/2020-01-18/failure-report/sel.html#software-evolution-library
I get this: Git failed: (:ABSOLUTE "home" "quicklisp" "quicklisp-controller" "dist" "build-cache" "sel" "ccbcc6b0d57db2cbc2c5290543ecfbed79846f25" "sel-20200118-git") is not in a git repository.
From the qlfile:
git cl-arrows [email protected]:research/cl-arrows.git
git elf [email protected]:research/elf.git
git testbot [email protected]:synthesis/testbot.git
git trace-db [email protected]:research/trace-db.git
git asdf-encodings [email protected]:synthesis/asdf-encodings.git
Are all of these open source? Can we move them to github, or point them to other forks instead?
Also, can we remove testbot?
software-evolution-library/run-new-clang-round-trip and
software-evolution-library/run-clang-diff-test fail for me.
Build log is here: http://report.quicklisp.org/2019-12-16/failure-report/sel.html
Unable to parse the single-file sqlite3.c "amalgamation." The file is attached sqlite3.c.txt. The result is a single error AST.
SEL/SW/C-CPP> (defparameter sqlite (from-file (make-instance 'c) "~/Projects/sqlite-bld/sqlite3.c"))
SQLITE
SEL/SW/C-CPP> (describe (@ sqlite 0))
#<C-ERROR 1217039 :TEXT "/*******************...">
[standard-object]
Slots with :CLASS allocation:
CHILD-SLOTS = ((CHILDREN . 0))
CHILD-SLOT-SPECIFIERS = (#<FUNCTIONAL-TREES::SLOT-SPECIFIER CHILDREN 0>)
Slots with :INSTANCE allocation:
STORED-HASH = NIL
OID = 1219463
DESCENDANT-MAP = #<unbound slot>
SERIAL-NUMBER = 1217039
SIZE = #<unbound slot>
ANNOTATIONS = NIL
BEFORE-TEXT = ""
TEXT = "/********************************************************************..
AFTER-TEXT = ""
BEFORE-ASTS = NIL
AFTER-ASTS = NIL
ORDERED-CHILDREN = NIL
INDENT-CHILDREN = NIL
INDENT-ADJUSTMENT = NIL
CHILDREN = NIL
; No value
Building with SBCL 2.0.5 / ASDF 3.3.1 for quicklisp dist creation.
Trying to build commit id e56fcf5
software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:
Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.
software-evolution-library/run-dump-store fails to build with the following error:
Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.
software-evolution-library/run-rest-server fails to build with the following error:
Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.
software-evolution-library/run-test-parse fails to build with the following error:
Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.
software-evolution-library/run-tree-sitter-interface fails to build with the following error:
Unhandled SB-PCL:CLASS-NOT-FOUND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: There is no class named SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER::C-FUNCTION-DEFINITION.
Hi there,
It is a great project, which makes up the transformation ability that the original tree-sitter haven't had yet.
Here I would like to perform a transformation that removing the type descriptor of the cast expression.
For example, given the input snippet as follows (which has two nested cast expressions in the corresponding AST):
void foo(byte *param)
{
sVar = (char *)xmalloc((int)param + 8);
}
, the expected output should be
void foo(byte *param)
{
sVar = xmalloc(param + 8);
}
To this end, I write the code as follows:
import asts
code = \
"""
void foo(byte *param)
{
sVar = (char *)xmalloc((int)param + 8);
}
"""
def remove_cast(ast: asts.AST):
if isinstance(ast, asts.CCastExpression):
print('TRANS:', ast.source_text, '->', ast.value.source_text)
return ast.value
def main():
root = asts.AST.from_string(code, asts.ASTLanguage.C, deepest=True)
new_root = asts.AST.transform(root, remove_cast)
print('New Code:\n', new_root.source_text)
if __name__ == '__main__':
main()
But the output is
TRANS: (char *)xmalloc((int)param + 8) -> xmalloc((int)param + 8)
TRANS: (int)param -> param
New Code:
param
, there is only an identifier in the generated code, which is unexpected.
After some trials, I found it can work well under the situation that a cast expression doesn't contain other cast expressions.
For example, if the input is
void foo(byte *param)
{
sVar = (char *)xmalloc(param + 8);
}
the output is
void foo(byte *param)
{
sVar = xmalloc(param + 8);
}
, which is expected.
Now I have no idea about how to deal with this problem... Can you help me please?
Thanks!
https://github.com/GrammaTech/sel/blob/master/utility/utility.lisp#L656 references sb-posix, but sb-posix is not listed as a dependency in the system file. That means loading it into sbcl will fail unless sb-posix is coincidentally already loaded.
As someone in the decompilation community, I'm very interested in doing my own research and exploration into this field. I was hoping some more specific information could be shared here (or elsewhere) about how sel and any other tools were leveraged in your research project.
Particularly, I am interested in performing a similar experiment but with MIPS-compiled c programs. If it makes more sense for me to reach out to someone privately and discuss elsewhere, happy to do that as well. Thanks in advance for your consideration.
I follow the installation instruction in this [https://grammatech.github.io/sel/Installation.html#Installation(https://grammatech.github.io/sel/Installation.html#Installation) to install the SEL library.
However, the instruction stated that cl-arrows recommends downloading from: https://github.com/eschulte/cl-arrows but it seems that the SEL library expects the cl-arrowz that can be downloaded from https://github.com/eschulte/cl-arrowz, which is a replacement of the cl-arrows library.
There are two possible ways to resolve this issue:
Have "test.py" python file:
class TestIndent:
"""\\
"""
Parse it:
import asts
with open("test.py") as f:
text = f.read()
ast = asts.AST.from_string(text, language=asts.ASTLanguage.Python)
with open("test.parsed.py", "w") as f:
f.write(ast.source_text)
The resulting "test.parsed.py" will have invalid indentation:
class TestIndent:
"""\\
"""
I get this: Unhandled SB-INT:INVALID-ARRAY-INDEX-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1000A18083}>: Invalid index 65 for (SIMPLE-VECTOR 1), should be a non-negative integer below 1.
Full log here: http://report.quicklisp.org/2020-05-30/failure-report/sel.html
Building with SBCL 2.0.5 / ASDF 3.3.1 for quicklisp dist creation.
Trying to build commit id e7a6ae7
software-evolution-library/run-dump-store fails to build with the following error:
Unhandled ASDF/FIND-COMPONENT:MISSING-DEPENDENCY in thread #<SB-THREAD:THREAD "main thread" RUNNING {10008D8083}>: Component "cl-tree-sitter" not found, required by #<PACKAGE-INFERRED-SYSTEM "software-evolution-library/software/tree-sitter">
software-evolution-library/run-rest-server fails to build with the following error:
Unhandled ASDF/FIND-COMPONENT:MISSING-DEPENDENCY in thread #<SB-THREAD:THREAD "main thread" RUNNING {10008D8083}>: Component "cl-tree-sitter" not found, required by #<PACKAGE-INFERRED-SYSTEM "software-evolution-library/software/tree-sitter">
Specifically I get the following error in my slime REPL. Commenting out the test-match-derive
test allows the file to load. This is with the versions of all tree-sitter libraries listed in tools/tree-sitter-install.sh
.
Error Backtrace:
; file: /Users/eschulte/quicklisp/local-projects/sel/test/template.lisp
; in: DEFTEST TEST-MATCH-DERIVE
; (TRIVIA.LEVEL2:EMATCH (SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::MAKE-DERIVE
; '("PartialEq" "PartialOrd"))
; ((SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER:RUST "#[derive(@ARGS)]"
; :ARGS
; SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS)
; (STEFIL:IS
; (FSET::EVERY #'SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/PARSEABLE:SOURCE-TEXT=
; SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS
; '("PartialEq" "PartialOrd")))))
; --> TRIVIA.LEVEL2:MATCH2+
; ==>
; (TRIVIA.LEVEL2:MATCH2*+ ((SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::MAKE-DERIVE
; '("PartialEq" "PartialOrd")))
; (T)
; (((SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/TREE-SITTER:RUST "#[derive(@ARGS)]"
; :ARGS
; SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS))
; (STEFIL:IS
; (FSET::EVERY #'SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/PARSEABLE:SOURCE-TEXT=
; SOFTWARE-EVOLUTION-LIBRARY/TEST/TEMPLATE::ARGS
; '("PartialEq" "PartialOrd"))))
; ((#:OTHERWISE1674)
; (ERROR 'TRIVIA.LEVEL2:MATCH-ERROR :PATTERN '((# #)) :VALUES
; (LIST #:OTHERWISE1674))))
;
; caught ERROR:
; during macroexpansion of
; (MATCH2*+ (#)
; (T)
; ...).
; Use *BREAK-ON-SIGNALS* to intercept.
;
; Unknown &KEY argument: :NAME-GENERATOR
Building with SBCL 2.2.7.28-02bc916fd / ASDF 3.3.5 for quicklisp dist creation.
Trying to build commit id 556deca
software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:
Unhandled SB-PCL::NO-APPLICABLE-METHOD-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10016D0003}>: There is no applicable method for the generic function #<COMMON-LISP:STANDARD-GENERIC-FUNCTION FSET:CONVERT (99)> when called with arguments (PYTHON-AST "from typing import Any, List
Building with SBCL 2.2.1 / ASDF 3.3.5 for quicklisp dist creation.
Trying to build commit id a8e0ef7
software-evolution-library/deploy-tree-sitter-interface fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
software-evolution-library/run-dump-store fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10418AC073}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
software-evolution-library/run-rest-server fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
software-evolution-library/run-test-parse fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
software-evolution-library/run-tree-sitter-interface fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
software-evolution-library/run-tree-sitter-py-generator fails to build with the following error:
; caught ERROR:
; during macroexpansion of (ITER (FOR AST IN-TREE ...) (MATCH AST #)). Use *BREAK-ON-SIGNALS* to intercept.
; Failed to deduce the way to access the slot C/CPP-PATH
...
Unhandled UIOP/LISP-BUILD:COMPILE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {1001B98003}>: COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "software-evolution-library/software/cpp" "file-type">
To load CFFI (and thus SEL) on a Mac M1 the latest changes from the main CFFI branch are required. Unfortunately death doesn't want to merge those into their fork of CFFI used by SEL. They suggest (see death/cffi#2 (comment)) maintaining a fork of CFFI with these changes and any others needed by SEL. You're welcome to use the fork I've started for this purpose (https://github.com/eschulte/cffi) or maybe you'd rather host one under GrammaTech on GitHub.
If you use the latest eclector, the following systems no longer build:
I get this:
USE-PACKAGE #<PACKAGE "ECLECTOR.PARSE-RESULT"> causes name-conflicts in #<PACKAGE "SOFTWARE-EVOLUTION-LIBRARY/SOFTWARE/LISP"> between the following symbols: ECLECTOR.PARSE-RESULT:READ-PRESERVING-WHITESPACE, COMMON-LISP:READ-PRESERVING-WHITESPACE
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.