soasme / peppapeg Goto Github PK
View Code? Open in Web Editor NEWPEG Parser in ANSI C
Home Page: https://soasme.com/PeppaPEG
License: MIT License
PEG Parser in ANSI C
Home Page: https://soasme.com/PeppaPEG
License: MIT License
Is your feature request related to a problem? Please describe.
Some programming languages, such as Go, allows unicode letters in identifiers.
Having a complete set of unicode letters in Peppa PEG grammar is tedious and low performant.
Describe the solution you'd like
Extend range to support [\p{L}]
in addition to current implementation: [a-f]
/ [\u{1}-\u{10ffff}]
.
Since \p itself has already the meaning of "range", there is no need to add a dash specifying the lower and upper.
More "\p" examples can be seen in Regex: https://www.regular-expressions.info/unicode.html, https://www.compart.com/en/unicode/category.
Describe alternatives you've considered
An alternative way is to provide built-in characters like in pest: UNICODE_LETTERS.
Additional context
N/A
Is your feature request related to a problem? Please describe.
Consider an HCL2 heredoc template,
a = <<EOT
something
EOT
We will need to determine if inner lines of Heredoc template is not started with EOT, hence
HeredocTemplate = "<<" Heredoc (!(\1 Newline) .)* \1 Newline;
As of v1.16.0, there is no way to get a back reference inside (...)*
as it's under its own Sequence scope.
See https://github.com/soasme/PeppaPEG/blob/main/configs/hcl2.peg.
Describe the solution you'd like
\M.N
, meaning Nth member in Mth member.Describe alternatives you've considered
N/A
Additional context
N/A
https://www.soasme.com/PeppaPEG/tutcalc.html
When reading the document, I see many C escape code, which makes the grammar a bit hard to read.
It looks like gcc support it. ( see this post: https://stackoverflow.com/questions/24850244/does-c-support-raw-string-literals )
Thanks.
Except creating a grammar object using P4_CreateXXX and P4_AddXXX APIs, it would be convenient to be able to create a grammar using PEG as input.
P4_String peg =
"entry = one one;\n"
"one = \"1\";"
P4_Grammar* grammar = P4_LoadGrammar(peg);
P4_Source* source = P4_Source("11", 1);
P4_Parse(grammar, source);
Peg/Leg, Pest.rs.
Is your feature request related to a problem? Please describe.
See discussion here
Describe the solution you'd like
Typedef const char* P4_CString and replace all types in source.
Describe alternatives you've considered
N/A
Additional context
N/A
Describe the bug
The provided tests/toml-v1.0.peg
can't capture some cases.
\uxxxx
/ \UXXXXXXXX
out of range. "Any Unicode character may be escaped with the \uXXXX or \UXXXXXXXX forms. The escape codes must be valid Unicode scalar values."Is your feature request related to a problem? Please describe.
See #137.
Many programming languages have a deep precedence levels, say Java has twentyish. If we use left recursion and right recursion, the performance would be so bad. Instead, many parsers such as GCC,Clang choose to implement precedence climbing.
Describe the solution you'd like
S = atom < op1 < op2 < op3 < op4;
. From a parsing's perspective, it's similar with S = atom ((op1 / op2 / op3 / op4) atom)*
, but the construction of AST is based on the precedence level.@right_associativity pow = "^";
. It allows operators using right associativity. For example, 1^2^3
should be parsed as 1^(2^3), while 1*2*3
should be parsed as (1*2)*3.Additional context
People writing grammars for https://github.com/tree-sitter/tree-sitter need to use a ugly JavaScript spaghetti to write grammars and convert then to a json that the actual parser uses to do it's job.
I've been looking for a simple way to write the grammars in a style like yacc/bison/peg and then the tool would write the final json for the tree-sitter parser, I think that PeppaPEG would be a nice fit for this job and for that we need read the actual json grammars and convert to an augmented EBNF then people would write/extend on that EBNF and the tool would again write back the json.
A feature like this one would improve the visibility and user base of PeppaPEG.
Example here is a Lua grammar for tree-sitter https://github.com/Azganoth/tree-sitter-lua/blob/master/grammar.js and here the final json https://github.com/Azganoth/tree-sitter-lua/blob/master/src/grammar.json and here an initial EBNF grammar :
%name lua
%extras
comment
/[\s\n]/
;
%conflicts
[ _prefix ]
[ _expression _variable_declarator ]
[ _expression function_call_statement ]
[ function_name function_name_field ]
;
%externals
comment
string
;
%inline
_statement
;
program : _statement* return_statement? ;
return_statement : "return" ( _expression ( "," _expression ) * ) ? _empty_statement? ;
_statement : expression | variable_declaration | local_variable_declaration | do_statement | if_statement | while_statement | repeat_statement | for_statement | for_in_statement | goto_statement | break_statement | label_statement | _empty_statement | function | local_function | function_call ;
variable_declaration : ( variable_declarator ( "," variable_declarator ) * ) "=" ( _expression ( "," _expression ) * ) ;
local_variable_declaration : "local" variable_declarator ( "=" ( _expression ( "," _expression ) * ) ) ? ;
_variable_declarator : identifier | ( _prefix "[" _expression "]" ) | field_expression ;
field_expression : _prefix "." property_identifier ;
_local_variable_declarator : identifier ( "," identifier ) * ;
do_statement : "do" _statement* return_statement? "end" ;
if_statement : "if" condition_expression "then" _statement* return_statement? elseif* else? "end" ;
elseif : "elseif" condition_expression "then" _statement* return_statement? ;
else : "else" _statement* return_statement? ;
while_statement : "while" condition_expression "do" _statement* return_statement? "end" ;
repeat_statement : "repeat" _statement* return_statement? "until" condition_expression ;
for_statement : "for" loop_expression "do" _statement* return_statement? "end" ;
for_in_statement : "for" loop_expression "do" _statement* return_statement? "end" ;
_loop_expression : identifier "=" _expression "," _expression ( "," _expression ) ? ;
_in_loop_expression : ( identifier ( "," identifier ) * ) "in" ( _expression ( "," _expression ) * ) ;
goto_statement : "goto" identifier ;
break_statement : "break" ;
label_statement : "::" identifier "::" ;
_empty_statement : ";" ;
function_statement : "function" function_name _function_body ;
local_function_statement : "local" "function" identifier _function_body ;
function_call_statement : ( ( ( _prefix arguments ) | ( _prefix ":" method arguments ) ) ) ;
arguments : ( "(" ( _expression ( "," _expression ) * ) ? ")" ) | table | string ;
function_name : ( identifier | function_name_field ) ( ":" method ) ? ;
function_name_field : ( identifier ) ( "." property_identifier ) * ;
parameters : "(" ( ( self | spread | identifier ) ( "," identifier ) * ( "," spread ) ? ) ? ")" ;
_function_body : parameters _statement* return_statement? "end" ;
_expression : spread | _prefix | next | function_definition | table | binary_operation | unary_operation | string | number | nil | true | false | identifier ;
spread : "..." ;
self : "self" ;
next : "next" ;
global_variable : "_G" | "_VERSION" ;
_prefix : self | global_variable | _variable_declarator | ( function_call ) | ( "(" _expression ")" ) ;
function_definition : "function" _function_body ;
table : "{" _field_sequence? "}" ;
field : ( "[" _expression "]" "=" _expression ) | ( identifier "=" _expression ) | _expression ;
_field_sequence : ( ( field ( _field_sep field ) * _field_sep? ) ) ;
_field_sep : "," | ";" ;
binary_operation : ( ( _expression "or" _expression ) ) | ( ( _expression "and" _expression ) ) | ( ( _expression "<" _expression ) ) | ( ( _expression "<=" _expression ) ) | ( ( _expression "==" _expression ) ) | ( ( _expression "~=" _expression ) ) | ( ( _expression ">=" _expression ) ) | ( ( _expression ">" _expression ) ) | ( ( _expression "|" _expression ) ) | ( ( _expression "~" _expression ) ) | ( ( _expression "&" _expression ) ) | ( ( _expression "<<" _expression ) ) | ( ( _expression ">>" _expression ) ) | ( ( _expression "+" _expression ) ) | ( ( _expression "-" _expression ) ) | ( ( _expression "*" _expression ) ) | ( ( _expression "/" _expression ) ) | ( ( _expression "//" _expression ) ) | ( ( _expression "%" _expression ) ) | ( ( _expression ".." _expression ) ) | ( ( _expression "^" _expression ) ) ;
unary_operation : ( ( ( "not" | "#" | "-" | "~" ) _expression ) ) ;
number : ( ( ( ( ( "0" | ( "0" ? /[1-9]/ /[0-9]+/? ) ) "." /[0-9]+/? ( ( "e" | "E" ) ( ( "-" | "+" ) ? /[0-9]+/ ) ) ? ) | ( "." /[0-9]+/ ( ( "e" | "E" ) ( ( "-" | "+" ) ? /[0-9]+/ ) ) ? ) | ( ( "0" | ( "0" ? /[1-9]/ /[0-9]+/? ) ) ( ( "e" | "E" ) ( ( "-" | "+" ) ? /[0-9]+/ ) ) ? ) ) | ( ( "0x" | "0X" ) /[a-fA-F0-9]+/ ( "." /[a-fA-F0-9]+/ ) ? ( ( "p" | "P" ) ( ( "-" | "+" ) ? /[0-9]+/ ) ) ? ) ) ) ;
nil : "nil" ;
true : "true" ;
false : "false" ;
identifier : /[a-zA-Z_][a-zA-Z0-9_]*/ ;
Generated by this script (using quickjs https://bellard.org/quickjs/):
import * as std from "std";
import * as os from "os";
//print(typeof process);
//if(typeof std == "undefined") {
//}
const fname_list = [
"tree-sitter-lua/src/grammar.json",
];
//const fname_base = "A_grammars/tree-sitter/";
const fname_base = "../tree-sitter/";
function parseJsonGrammar(fname)
{
let fd = std.open(fname_base + fname, "r");
let json = fd.readAsString();
fd.close();
let out_fname = fname.replace(/\/.+/, ".ebnf0");
//print(out_fname); return;
//fd = std.open(out_fname, "w");
fd = std.open("/dev/stdout", "w"); //std.stdout;
json = JSON.parse(json);
//print(json);
fd.printf("%%name %s\n", json.name);
if(json.word) fd.printf("%%word %s\n", json.word);
let manageTuples = function(dict) {
for(var idx in dict) {
let value = dict[idx];
//print(value);
switch(value.type) {
case "SYMBOL":
fd.printf("\t%s\n", value.name);
break;
case "PATTERN":
fd.printf("\t/%s/\n", value.value);
break;
case "STRING":
fd.printf("\t'%s'\n", value.value);
break;
case "TOKEN":
fd.printf("\t'%s'\n", value.content);
break;
default:
throw("Unmanaged type " + value.type);
}
}
};
let manageArrays = function(ary) {
for(var idx in ary) {
let value = ary[idx];
fd.printf("\t[");
for(var idx2 in value) {
fd.printf(" %s", value[idx2]);
}
fd.printf(" ]\n");
}
};
if( json.extras && json.extras.length ) {
fd.printf("%%extras\n");
manageTuples(json.extras);
fd.printf("\t;\n");
}
if( json.conflicts && json.conflicts.length ) {
fd.printf("%%conflicts\n");
manageArrays(json.conflicts);
fd.printf("\t;\n");
}
if( json.externals && json.externals.length ) {
fd.printf("%%externals\n");
manageTuples(json.externals);
fd.printf("\t;\n");
}
if( json.precedences && json.precedences.length) {
fd.printf("%%precedences\n");
for(var idx in json.precedences) {
let value = json.precedences[idx];
fd.printf("\t[");
for(var idx2 in value) {
let pval = value[idx2];
switch(pval.type) {
case "SYMBOL":
fd.printf(" %s", pval.name);
break;
case "STRING":
fd.printf(" '%s'", pval.value);
break;
default:
throw("Unmanaged type " + pval.type);
}
}
fd.printf(" ]\n");
}
fd.printf("\t;\n");
}
if( json.inline && json.inline.length ) {
fd.printf("%%inline\n");
for(var idx in json.inline) {
let value = json.inline[idx];
fd.printf("\t%s\n", value);
}
fd.printf("\t;\n");
}
if( json.supertypes && json.supertypes.length ) {
fd.printf("%%supertypes\n");
for(var idx in json.supertypes) {
let value = json.supertypes[idx];
fd.printf("\t%s\n", value);
}
fd.printf("\t;\n");
}
let manageAlias = function (rules, alias_list) {
for(var idx in rules) {
let rhs = rules[idx2];
for(var idx2 in rhs) {
}
}
};
let alias_list = {};
let str_list = [];
let manageRule = function (name, rule, depth) {
//print(name, rule.type); //, typeof rule);
switch(rule.type)
{
case "ALIAS":
//fd.printf(" ( ");
//manageRule(rule.type, rule.content, depth+1);
//fd.printf(" )@%s ", rule.value);
fd.printf("%s", rule.value);
//alias_list[rule.value] = ;
break;
case "BLANK":
print(rule.type);
break;
case "CHOICE": {
let members = rule.members;
let mcount = members.length;
let isOptional = mcount == 2 && members[1].type == "BLANK";
if(isOptional) {
//if(mcount > 2 && depth) fd.printf(" (");
manageRule(rule.type, members[0], depth+1);
//if(mcount > 2 && depth) fd.printf(" )");
fd.printf("? ");
}
else
{
//fd.printf("=== %d : %d : %d ===", mcount, depth, mcount > 1 && depth);
if(mcount > 1 && depth) fd.printf(" (");
for(var idx in members) {
if(idx > 0) fd.printf(" | ");
manageRule(rule.type, members[idx], depth+1);
}
if(mcount > 1 && depth) fd.printf(" ) ");
}
}
break;
case "FIELD":
//print(rule.type, rule.name);
fd.printf(" ( ");
manageRule(rule.type, rule.content, depth+1);
fd.printf(" ) ");
break;
case "IMMEDIATE_TOKEN":
fd.printf(" ( ");
manageRule(rule.type, rule.content, depth+1);
fd.printf(" ) ");
break;
case "PATTERN": {
fd.printf(" /%s/", rule.value);
}
break;
case "PREC":
case "PREC_DYNAMIC":
case "PREC_LEFT":
case "PREC_RIGHT":
fd.printf(" ( ");
manageRule(rule.type, rule.content, depth+1);
fd.printf(" ) ");
break;
case "REPEAT":
//if(depth) fd.printf(" ( ");
manageRule(rule.type, rule.content, depth+1);
//if(depth) fd.printf(" )");
fd.printf("* ");
break;
case "REPEAT1":
//if(depth) fd.printf(" (");
manageRule(rule.type, rule.content, depth+1);
//if(depth) fd.printf(" )");
fd.printf("+ ");
break;
case "SEQ": {
let members = rule.members;
let mcount = members.length;
//fd.printf("=== %d : %d ===", mcount, depth);
if(mcount > 1 && depth) fd.printf(" ( ");
for(var idx in members) {
manageRule(rule.type, members[idx], depth+1);
}
if(mcount > 1 && depth) fd.printf(" ) ");
}
break;
case "STRING": {
let value = rule.value;
//print(rule.type, value);
switch(value) {
case "\0": fd.printf(" '\\0' "); break;
case "\b": fd.printf(" '\\b' "); break;
case "\f": fd.printf(" '\\f' "); break;
case "\n": fd.printf(" '\\n' "); break;
case "\r": fd.printf(" '\\r' "); break;
case "\t": fd.printf(" '\\t' "); break;
case "\v": fd.printf(" '\\v' "); break;
case "\\": fd.printf(" '\\\\' "); break;
case "'": fd.printf(" \"'\" "); break;
case "\"": fd.printf(" '\"' "); break;
default:
//value = value.replace(/\\/g, "\\\\");
value = value.replace(/\t/g, "\\t"); //order matter
if(value.indexOf("'") >= 0) fd.printf(" \"%s\" ", value);
else fd.printf(" \"%s\" ", value);
}
}
break;
case "SYMBOL":
//print(rule.type, rule.name);
fd.printf(" %s", rule.name);
break;
case "TOKEN":
fd.printf(" ( ");
manageRule(rule.type, rule.content, depth+1);
fd.printf(" ) ");
break;
default:
throw("Unknown rule type: " + rule.type);
}
}
let rules = json.rules;
for(var idx in rules) {
let rule = rules[idx];
//print(rule);
if(rule.type == "xTOKEN") {
let str = idx + " : " + + " .";
}
else {
fd.printf("%s : ", idx);
manageRule(idx, rule, 0);
}
fd.printf(" ;\n");
}
fd.close();
}
for(let idx in fname_list)
{
let fname = fname_list[idx];
print(fname);
parseJsonGrammar(fname);
}
Describe the bug
Segfault when grammar entry rule doesn't exists
To Reproduce
Steps to reproduce the behavior:
README
json example execute peppa parse -G json.peg -e entry2 data.json
Desktop (please complete the following information):
This seems to fix the problem (line numbers are from my code with other changes):
@@ -3194,9 +3330,9 @@ P4_GetErrorMessage(P4_Source* source) {
return NULL;
memset(source->errmsg, 0, sizeof(source->errmsg));
sprintf(source->errmsg, "line %zu:%zu, expect %s",
source->error.lineno, source->error.offset,
- source->error.rule->name);
+ source->error.rule ? source->error.rule->name : source->entry_name);
Proposed to name the library as python-peppapeg
.
Example usage:
>>> import peppapeg as P4
>>> grammar = P4.Grammar()
>>> grammar.add_sequence('entry', 'greeter', grammar.literal(' '), 'target')
>>> grammar.add_literal('greeter', 'Hello')
>>> grammar.add_literal('target', 'world')
>>> ast = grammar.parse("Hello world")
>>> print(ast)
Token(0..11)
With something like this mingodad@d8e43ed and using https://www.bottlecaps.de/convert/ to convert the naked
output to be used here https://www.bottlecaps.de/rr/ui to get a railroad diagram for the grammars written to PeppaPEG.
Example:
../cli parse -n -G peppa.peg -e grammar peppa.peg
grammar =
( start_of_input rule+ end_of_input ) ;
start_of_input =
&. ;
end_of_input =
!. ;
rule =
( decorators name /*~*/ "=" expression ";" ) ;
decorators =
decorator* ;
decorator =
( "@" /*~*/ ( "squashed" / "scoped" / "spaced" / "lifted" / "tight" / "nonterminal" ) ) ;
/*@squashed*/
name =
reference ;
/*@lifted*/
expression =
left_recursion ;
/*@nonterminal*/
left_recursion =
( choice ( "|" /*~*/ reference choice )? ) ;
/*@nonterminal*/
choice =
( sequence ( ! ( "//" / "/*" ) "/" sequence )* ) ;
/*@nonterminal*/
sequence =
repeat+ ;
/*@nonterminal*/
repeat =
( primary ( onceormore / zeroormore / zerooronce / repeatexact / repeatminmax / repeatmin / repeatmax )? ) ;
onceormore =
"+" ;
zeroormore =
"*" ;
zerooronce =
"?" ;
repeatexact =
( "{" number "}" ) ;
repeatminmax =
( "{" number "," number "}" ) ;
repeatmin =
( "{" number "," "}" ) ;
repeatmax =
( "{" "," number "}" ) ;
/*@lifted*/
primary =
( literal / insensitive / range / ( reference !"=" ) / back_reference / positive / negative / ( "(" choice ")" ) / dot / cut ) ;
/*@squashed*/ /*@tight*/
literal =
( "\"" /*~*/ chars "\"" ) ;
chars =
char* ;
/*@squashed*/ /*@tight*/
char =
( [\x20-\x21] / [\x23-\x5b] / [\x5d-\U0010ffff] / ( "\\" ( "\"" / "/" / "\\" / "b" / "f" / "n" / "r" / "t" / "v" / ( "x" /*~*/ two_hexdigits ) / ( "u" /*~*/ four_hexdigits ) / ( "U" /*~*/ eight_hexdigits ) ) ) ) ;
/*@squashed*/ /*@tight*/
range_category =
( [a-z] / [A-Z] / [0-9] / "_" / " " )+ ;
range =
( "[" ( ( char "-" char ( ".." number )? ) / ( "\\p{" range_category "}" ) ) "]" ) ;
/*@tight*/
insensitive =
( "i" ( literal / back_reference ) ) ;
/*@squashed*/ /*@tight*/
reference =
( ( [a-z] / [A-Z] / "_" ) ( [a-z] / [A-Z] / [0-9] / "_" )* ) ;
/*@tight*/
back_reference =
( "\\" /*~*/ number ) ;
positive =
( "&" /*~*/ primary ) ;
negative =
( "!" /*~*/ primary ) ;
dot =
"." ;
cut =
"~" ;
hexdigit =
( [0-9] / [a-f] / [A-F] ) ;
two_hexdigits =
hexdigit{2} ;
four_hexdigits =
hexdigit{4} ;
eight_hexdigits =
hexdigit{8} ;
/*@squashed*/ /*@tight*/
number =
( "0" / ( [1-9] [0-9]* ) ) ;
/*@lifted*/ /*@spaced*/
comment =
( ( ( "#" / "//" ) ( !"\n". )* "\n"? ) / ( "/*" ( !"*/". )* "*/" ) ) ;
/*@lifted*/ /*@spaced*/
whitespace =
( " " / "\t" / "\r" / "\n" ) ;
Then coping and pasting the output shown above on https://www.bottlecaps.de/convert/ on the Input grammar:
textarea and then clicking the button Convert
then the button View Diagram
we get a nice railroad diagram (https://en.wikipedia.org/wiki/Syntax_diagram) for the grammar.
Describe the bug
When building with make check
several tests fail
To Reproduce
Steps to reproduce the behavior:
Obs: I've added two print
on check_spec.y
:
if 'grammar' in spec:
cmd.extend(['--grammar-str', spec['grammar']])
elif 'grammar_file' in spec:
if spec['grammar_file'].startswith('/'):
cmd.extend(['--grammar-file', spec['grammar_file']])
else:
cmd.extend(['--grammar-file', os.path.dirname(os.path.abspath(specs_file)) + '/' + spec['grammar_file']])
else:
raise ValueError('Missing grammar/grammar_file')
print(cmd) #<<<< here
print(test['I'].encode('utf-8')) #<<<< here
proc = subprocess.run(
cmd,
capture_output=True,
input=test['I'].encode('utf-8'),
)
make check
[ 3%] Built target unity
[ 6%] Built target peppa
[ 11%] Built target test_flags
[ 15%] Built target cli
[ 20%] Built target test_example_dot
[ 25%] Built target test_example_calc
[ 30%] Built target test_example_json
[ 35%] Built target test_peg
[ 40%] Built target test_positive
[ 45%] Built target test_literal
[ 50%] Built target test_range
[ 55%] Built target test_misc
[ 60%] Built target test_cut
[ 65%] Built target test_sequence
[ 70%] Built target test_choice
[ 75%] Built target test_reference
[ 80%] Built target test_negative
[ 85%] Built target test_example_mustache
[ 90%] Built target test_repeat
[ 95%] Built target test_left_recursion
[100%] Built target test_back_reference
Test project PeppaPEG/build/tests
Start 1: test_literal
1/48 Test #1: test_literal ..................... Passed 0.00 sec
Start 2: test_range
2/48 Test #2: test_range ....................... Passed 0.00 sec
Start 3: test_sequence
3/48 Test #3: test_sequence .................... Passed 0.00 sec
Start 4: test_choice
4/48 Test #4: test_choice ...................... Passed 0.00 sec
Start 5: test_reference
5/48 Test #5: test_reference ................... Passed 0.00 sec
Start 6: test_positive
6/48 Test #6: test_positive .................... Passed 0.00 sec
Start 7: test_negative
7/48 Test #7: test_negative .................... Passed 0.00 sec
Start 8: test_repeat
8/48 Test #8: test_repeat ...................... Passed 0.00 sec
Start 9: test_flags
9/48 Test #9: test_flags ....................... Passed 0.00 sec
Start 10: test_cut
10/48 Test #10: test_cut ......................... Passed 0.00 sec
Start 11: test_back_reference
11/48 Test #11: test_back_reference .............. Passed 0.00 sec
Start 12: test_left_recursion
12/48 Test #12: test_left_recursion .............. Passed 0.00 sec
Start 13: test_misc
13/48 Test #13: test_misc ........................ Passed 0.00 sec
Start 14: test_peg
14/48 Test #14: test_peg ......................... Passed 0.01 sec
Start 15: test_example_mustache
15/48 Test #15: test_example_mustache ............ Passed 0.00 sec
Start 16: test_example_json
16/48 Test #16: test_example_json ................ Passed 0.02 sec
Start 17: test_example_calc
17/48 Test #17: test_example_calc ................ Passed 0.01 sec
Start 18: test_example_dot
18/48 Test #18: test_example_dot ................. Passed 0.08 sec
Start 19: test_literal_spec.json
19/48 Test #19: test_literal_spec.json ...........***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = "1";']
b'1'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 20: test_range_spec.json
20/48 Test #20: test_range_spec.json .............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'Num', '--grammar-str', 'Num = [1-9];']
b'1'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 21: test_insensitive_spec.json
21/48 Test #21: test_insensitive_spec.json .......***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = i"1";']
b'1'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 22: test_sequence_spec.json
22/48 Test #22: test_sequence_spec.json ..........***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = "a" "b" "c";']
b'abc'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 23: test_choice_spec.json
23/48 Test #23: test_choice_spec.json ............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = "a" / "b" / "c";']
b'a'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 24: test_left_recursion_spec.json
24/48 Test #24: test_left_recursion_spec.json ....***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'A', '--grammar-str', 'A = "a" | B "a"; B = "b";']
b'a'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 25: test_lookahead_spec.json
25/48 Test #25: test_lookahead_spec.json .........***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = !"a" [a-f];']
b'b'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 26: test_cut_spec.json
26/48 Test #26: test_cut_spec.json ...............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = R2*;R2 = "1" ~ "2";']
b'121'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 27: test_repeat_spec.json
27/48 Test #27: test_repeat_spec.json ............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = ([0-9] / [a-f] / [A-F])+;']
b'1A9F'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 28: test_reference_spec.json
28/48 Test #28: test_reference_spec.json .........***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = R2; R2 = "1";']
b'1'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 29: test_back_reference_spec.yaml
29/48 Test #29: test_back_reference_spec.yaml ....***Failed 0.12 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = R23 \\0;\nR23 = "2" / "3";\n']
b'22'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 30: test_dot_spec.json
30/48 Test #30: test_dot_spec.json ...............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = .;']
b'\xe5\xa5\xbd'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 31: test_comment_spec.json
31/48 Test #31: test_comment_spec.json ...........***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', 'R1 = "1"# R1 = "3";\n;']
b'1'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 32: test_flags_spec.json
32/48 Test #32: test_flags_spec.json .............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'R1', '--grammar-str', '@squashed\nR1 = R2 R2;R2 = "X";']
b'XX'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 33: test_peppa_spec.json
33/48 Test #33: test_peppa_spec.json .............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'number', '--grammar-file', 'PeppaPEG/tests/./peppa.peg']
b'0'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 34: test_golang_v1_17_spec.json
34/48 Test #34: test_golang_v1_17_spec.json ......***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'Literal', '--grammar-file', 'PeppaPEG/tests/../configs/golang-v1.17.peg']
b'42'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 35: toml/test_array.yaml
35/48 Test #35: toml/test_array.yaml .............***Failed 0.22 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'ints = [1, 2, 3, ]\nfloats = [1.1, 2.1, 3.1]\nstrings = ["a", "b", "c"]\ndates = [\n 1987-07-05T17:45:00Z,\n 1979-05-27T07:32:00Z,\n 2006-06-01T11:00:00Z,\n]\ncomments = [\n 1,\n 2, #this is ok\n]\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 36: toml/test_bool.yaml
36/48 Test #36: toml/test_bool.yaml ..............***Failed 0.11 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b't = true\nf = false\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 37: toml/test_comment.yaml
37/48 Test #37: toml/test_comment.yaml ...........***Failed 0.16 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'# This is a full-line comment\nkey = "value" # This is a comment at the end of a line\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 38: toml/test_control.yaml
38/48 Test #38: toml/test_control.yaml ...........***Failed 0.12 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'comment-lf = "ctrl-P" # \x10\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 39: toml/test_datetime.yaml
39/48 Test #39: toml/test_datetime.yaml ..........***Failed 0.21 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'space = 1987-07-05 17:45:00Z\nlower = 1987-07-05t17:45:00z\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 40: toml/test_float.yaml
40/48 Test #40: toml/test_float.yaml .............***Failed 0.17 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'double-point-1 = 0..1\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 41: toml/test_inline_table.yaml
41/48 Test #41: toml/test_inline_table.yaml ......***Failed 0.33 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'people = [{first_name = "Bruce", last_name = "Springsteen"},\n {first_name = "Eric", last_name = "Clapton"},\n {first_name = "Bob", last_name = "Seger"}]\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 42: toml/test_integer.yaml
42/48 Test #42: toml/test_integer.yaml ...........***Failed 0.17 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'capital-bin = 0B0\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 43: toml/test_key.yaml
43/48 Test #43: toml/test_key.yaml ...............***Failed 0.27 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'[[agencies]] owner = "S Cjelli"\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 44: toml/test_string.yaml
44/48 Test #44: toml/test_string.yaml ............***Failed 0.24 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'naughty = "\\xAg"\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 45: toml/test_table.yaml
45/48 Test #45: toml/test_table.yaml .............***Failed 0.24 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'toml', '--grammar-file', 'PeppaPEG/tests/toml/../../configs/toml-v1.0.peg']
b'[[]]\nname = "Born to Run"\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 46: abnf/test_rfc5234.yaml
46/48 Test #46: abnf/test_rfc5234.yaml ...........***Failed 0.24 sec
['PeppaPEG/build/cli', 'parse', '--grammar-entry', 'RuleList', '--grammar-file', 'PeppaPEG/tests/abnf/../../configs/abnf.peg']
b'rulelist = 1*( rule / (*c-wsp c-nl) )\n\nrule = rulename defined-as elements c-nl\n ; continues if next line starts\n ; with white space\n\nrulename = ALPHA *(ALPHA / DIGIT / "-")\n\ndefined-as = *c-wsp ("=" / "=/") *c-wsp\n ; basic rules definition and\n ; incremental alternatives\n\nelements = alternation *c-wsp\n\nc-wsp = WSP / (c-nl WSP)\n\nc-nl = comment / CRLF\n ; comment or newline\n\ncomment = ";" *(WSP / VCHAR) CRLF\n\nalternation = concatenation\n *(*c-wsp "/" *c-wsp concatenation)\n\nconcatenation = repetition *(1*c-wsp repetition)\n\nrepetition = [repeat] element\n\nrepeat = 1*DIGIT / (*DIGIT "*" *DIGIT)\n\nelement = rulename / group / option /\n char-val / num-val / prose-val\n\ngroup = "(" *c-wsp alternation *c-wsp ")"\n\noption = "[" *c-wsp alternation *c-wsp "]"\n\nchar-val = DQUOTE *(%x20-21 / %x23-7E) DQUOTE\n ; quoted string of SP and VCHAR\n ; without DQUOTE\n\nnum-val = "%" (bin-val / dec-val / hex-val)\n\nbin-val = "b" 1*BIT\n [ 1*("." 1*BIT) / ("-" 1*BIT) ]\n ; series of concatenated bit values\n ; or single ONEOF range\n\ndec-val = "d" 1*DIGIT\n [ 1*("." 1*DIGIT) / ("-" 1*DIGIT) ]\n\nhex-val = "x" 1*HEXDIG\n [ 1*("." 1*HEXDIG) / ("-" 1*HEXDIG) ]\n\nprose-val = "<" *(%x20-3D / %x3F-7E) ">"\n ; bracketed string of SP and VCHAR\n ; without angles\n ; prose description, to be used as\n ; last resort\n'
Traceback (most recent call last):
File "PeppaPEG/scripts/check_spec.py", line 87, in <module>
test_spec()
File "PeppaPEG/scripts/check_spec.py", line 48, in test_spec
input=test['I'].encode('utf-8'),
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
Start 47: lua-5.4.3/*.lua
47/48 Test #47: lua-5.4.3/*.lua .................. Passed 2.31 sec
Start 48: abnf/*.abnf
48/48 Test #48: abnf/*.abnf ...................... Passed 0.05 sec
42% tests passed, 28 tests failed out of 48
Total Test time (real) = 6.75 sec
The following tests FAILED:
19 - test_literal_spec.json (Failed)
20 - test_range_spec.json (Failed)
21 - test_insensitive_spec.json (Failed)
22 - test_sequence_spec.json (Failed)
23 - test_choice_spec.json (Failed)
24 - test_left_recursion_spec.json (Failed)
25 - test_lookahead_spec.json (Failed)
26 - test_cut_spec.json (Failed)
27 - test_repeat_spec.json (Failed)
28 - test_reference_spec.json (Failed)
29 - test_back_reference_spec.yaml (Failed)
30 - test_dot_spec.json (Failed)
31 - test_comment_spec.json (Failed)
32 - test_flags_spec.json (Failed)
33 - test_peppa_spec.json (Failed)
34 - test_golang_v1_17_spec.json (Failed)
35 - toml/test_array.yaml (Failed)
36 - toml/test_bool.yaml (Failed)
37 - toml/test_comment.yaml (Failed)
38 - toml/test_control.yaml (Failed)
39 - toml/test_datetime.yaml (Failed)
40 - toml/test_float.yaml (Failed)
41 - toml/test_inline_table.yaml (Failed)
42 - toml/test_integer.yaml (Failed)
43 - toml/test_key.yaml (Failed)
44 - toml/test_string.yaml (Failed)
45 - toml/test_table.yaml (Failed)
46 - abnf/test_rfc5234.yaml (Failed)
Errors while running CTest
tests/CMakeFiles/check.dir/build.make:94: recipe for target 'tests/CMakeFiles/check' failed
make[3]: *** [tests/CMakeFiles/check] Error 8
CMakeFiles/Makefile2:309: recipe for target 'tests/CMakeFiles/check.dir/all' failed
make[2]: *** [tests/CMakeFiles/check.dir/all] Error 2
CMakeFiles/Makefile2:316: recipe for target 'tests/CMakeFiles/check.dir/rule' failed
make[1]: *** [tests/CMakeFiles/check.dir/rule] Error 2
Makefile:248: recipe for target 'check' failed
make: *** [check] Error 2
Is your feature request related to a problem? Please describe.
In some cases, embedding range pattern tables load a lot of data into memory and it's unnecessary.
It's up to the user to decide whether to enable range patterns.
Describe the solution you'd like
Add a new macro: ENABLE_UNISTR
.
If it's enabled, peppapeg will load unicode categories from libunistring, otherwise, [\p{..}]
only supports some built-in categories.
Describe alternatives you've considered
N/A
Additional context
When trying to create a grammar with PeppaPEG I was not getting the expected result and the error message didn't helped me at first then I decided to add a debug info/trace to help me see where the parser was at the time of the error, I saw the already reserved P4_Source.verbose
flag after I added the trace code (see attaced diff, ignore the changes for the Lua grammar) and I'm leaving it here just in case it can be useful to others or even be included directly tho this project:
Partial output of peppa parse -d -G json.peg -e entry data.json
:
===> recursion_depth peppa.c:function -> rule_name : input_line_number
===> 0 P4_Parse -> : 1
===> 1 match_expression -> : 1
===> 2 match_sequence -> entry : 1
===> 3 match_expression -> entry : 1
===> 4 match_positive -> entry : 1
===> 5 match_expression -> entry : 1
===> 6 match_range -> entry : 1
===> 4 match_expression -> entry : 1
===> 5 match_repeat -> entry : 1
===> 6 match_expression -> entry : 1
===> 7 match_choice -> entry : 1
===> 8 match_expression -> entry : 1
===> 9 match_reference -> entry : 1
===> 10 match_expression -> entry : 1
===> 11 match_choice -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 3 match_expression -> entry : 1
===> 4 match_reference -> entry : 1
===> 5 match_expression -> entry : 1
===> 6 match_choice -> value : 1
===> 7 match_expression -> value : 1
===> 8 match_reference -> value : 1
===> 9 match_expression -> value : 1
===> 10 match_sequence -> object : 1
===> 11 match_expression -> object : 1
===> 12 match_literal -> object : 1
...
===> 4 match_expression -> entry : 1
===> 5 match_repeat -> entry : 1
===> 6 match_expression -> entry : 1
===> 7 match_choice -> entry : 1
===> 8 match_expression -> entry : 1
===> 9 match_reference -> entry : 1
===> 10 match_expression -> entry : 1
===> 11 match_choice -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 12 match_expression -> whitespace : 1
===> 13 match_literal -> whitespace : 1
===> 3 match_expression -> entry : 2
===> 4 match_negative -> entry : 2
===> 5 match_expression -> entry : 2
===> 6 match_range -> entry : 2
[{"slice":[0,50],"type":"array","children":[{"slice":[1,25],"type":"object","children":[{"slice":[2,24],"type":"item","children":[{"slice":[2,11],"type":"string"},{"slice":[13,24],"type":"array","children":[{"slice":[14,15],"type":"number"},{"slice":[16,19],"type":"number"},{"slice":[20,23],"type":"number"}]}]}]},{"slice":[26,43],"type":"array","children":[{"slice":[27,31],"type":"true"},{"slice":[32,37],"type":"false"},{"slice":[38,42],"type":"null"}]},{"slice":[44,49],"type":"string"}]}]
PeppaPEG version: 1.2.0.
Given program, half of the execution time spent on NeedLoosen. We should optimize it.
$ cat examples/json.c
#include <stdio.h>
#include <stdlib.h>
#include "../peppapeg.h"
#include "json.h"
# define NESTING_DEPTH 1000
int main(int argc, char* argv[]) {
char* input = malloc(sizeof(char) * (NESTING_DEPTH*2 + 1));
int i;
for (i = 0; i < NESTING_DEPTH; i++) {
input[i] = '[';
input[NESTING_DEPTH+i] = ']';
}
input[NESTING_DEPTH*2] = '\0';
P4_Grammar* grammar = P4_CreateJSONGrammar();
P4_Source* source = P4_CreateSource(input, P4_JSONEntry);
printf("%u\n", P4_Parse(grammar, source));
free(input);
P4_DeleteSource(source);
P4_DeleteGrammar(grammar);
return 0;
}
Run it with valgrind:
$ gcc -g peppapeg.c examples/json.c && valgrind --tool=callgrind ./a.out
$ callgrind_annotate callgrind.out.63642
We have below profiling result:
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
1,265,470,759 (100.0%) PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
706,187,168 (55.80%) peppapeg.c:P4_NeedLoosen [/app/a.out]
320,620,450 (25.34%) peppapeg.c:P4_IsTight [/app/a.out]
170,320,160 (13.46%) peppapeg.c:P4_IsScoped [/app/a.out]
25,020,000 ( 1.98%) peppapeg.c:P4_NeedSquash [/app/a.out]
10,000,000 ( 0.79%) peppapeg.c:P4_IsSquashed [/app/a.out]
4,617,160 ( 0.36%) ???:_int_free [/usr/lib64/libc-2.28.so]
3,382,226 ( 0.27%) ???:malloc [/usr/lib64/ld-2.28.so]
2,727,160 ( 0.22%) peppapeg.c:P4_Match'2 [/app/a.out]
1,925,056 ( 0.15%) ???:__strlen_avx2 [/usr/lib64/libc-2.28.so]
1,848,485 ( 0.15%) ???:free [/usr/lib64/ld-2.28.so]
1,841,351 ( 0.15%) peppapeg.c:P4_MatchLiteral [/app/a.out]
1,705,988 ( 0.13%) peppapeg.c:P4_MatchChoice'2 [/app/a.out]
1,540,594 ( 0.12%) peppapeg.c:P4_IsRule [/app/a.out]
1,518,598 ( 0.12%) peppapeg.c:P4_Expression_dispatch'2 [/app/a.out]
$ time ./a.out
real 0m0.157s
user 0m0.154s
sys 0m0.002s
A full profiling result can be seen here: https://gist.github.com/soasme/38471063511aa14302e5ccad173767de
Describe the bug
In the example on the readme, it fails with the message MatchError: line 1:1, expect value
.
To Reproduce
❯ cat json.peg
@lifted entry = &. value !.;
@lifted value = object / array / string / number / true / false / null;
object = "{" (item ("," item)*)? "}";
item = string ":" value;
array = "[" (value ("," value)*)? "]";
@tight string = "\"" ([\u0020-\u0021] / [\u0023-\u005b] / [\u005d-\U0010ffff] / escape )* "\"";
true = "true";
false = "false";
null = "null";
@tight @squashed
number = minus? integral fractional? exponent?;
@tight @squashed @lifted
escape = "\\" ("\"" / "/" / "\\" / "b" / "f" / "n" / "r" / "t" / unicode);
@tight @squashed
unicode = "u" ([0-9] / [a-f] / [A-F]){4};
minus = "-";
plus = "+";
@squashed @tight
integral = "0" / [1-9] [0-9]*;
@squashed @tight
fractional = "." [0-9]+;
@tight
exponent = i"e" (plus / minus)? [0-9]+;
@spaced @lifted
whitespace = " " / "\r" / "\n" / "\t";
❯ cat data.json
[{"numbers": [1,,2.0,3e1]},[true,false,null],"xyz"]
❯ ./peppa parse -G json.peg -e entry data.json
data.json:
MatchError: line 1:1, expect value
Expected behavior
A successful parse
Desktop (please complete the following information):
Additional context
❯ ./peppa -V
./peppa version 1.16.0
Describe the bug
error on compiling the header file
To Reproduce
PS C:\Users\risharan\DevSpace\c\wrench> make
gcc -O2 -Wall -Wall -Wextra -Werror -Wpedantic -std=c11 .\src\wrench.c -o .\bin\wrench
In file included from .\src\wrench.c:4:
.\src\../deps/peppa.h:1817:1: error: type qualifiers ignored on function return type [-Werror=ignored-qualifiers]
const P4_String P4_GetRuleName(P4_Expression* expr);
^~~~~
cc1.exe: all warnings being treated as errors
make: *** [Makefile:5: run] Error 1
Expected behavior
the line 1817 should not be there? on commenting it, the error goes away.
Screenshots
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
Comparing performance of json parser with Lua show that PeppaPEG is 2x memory hungry and takes 2x more time.
Describe the solution you'd like
At least the same performance or better.
Here is the Lua script using dkjson (http://dkolf.de/src/dkjson-lua.fsl/home):
local json = require ("dkjson")
local str = io.open(arg[1]):read("a*")
local obj, pos, err = json.decode (str, 1, nil)
if err then
print ("Error:", err)
else
for k,v in pairs(obj) do
print (k, type(v))
end
end
And here is the script testing all with this file as imput (https://github.com/rust-lang/rls/blob/master/rls-analysis/test_data/rls-analysis/librls_data.json 7MB) :
#!/bin/sh
fname="rustc-1.49.0-src/src/tools/rls/rls-analysis/test_data/rls-analysis/librls_data.json"
/usr/bin/time lua decode-json-file.lua $fname
/usr/bin/time lua decode-json-file-lpeg.lua $fname
/usr/bin/time PeppaPEG-DAD/cli parse -G json.peg -e entry $fname > /dev/null
And here is the output:
./test-big.sh
compilation table
refs table
config table
defs table
imports table
version string
relations table
impls table
prelude table
macro_refs table
1.23user 0.00system 0:01.23elapsed 100%CPU (0avgtext+0avgdata 59844maxresident)k
0inputs+0outputs (0major+16432minor)pagefaults 0swaps
config table
imports table
defs table
compilation table
refs table
impls table
macro_refs table
version string
relations table
prelude table
0.37user 0.01system 0:00.38elapsed 99%CPU (0avgtext+0avgdata 37248maxresident)k
0inputs+0outputs (0major+10731minor)pagefaults 0swaps
0 true
2.87user 0.02system 0:02.89elapsed 99%CPU (0avgtext+0avgdata 124372maxresident)k
0inputs+0outputs (0major+30745minor)pagefaults 0swaps
Hi,
While I was fuzzing library, I encountered some bugs. You can find them in below.
#include <stdio.h>
#include "/mnt/ramdisk/PeppaPEG/peppapeg.h"
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <string.h>
# define ENTRY 1
int main(int argc, char* argv[]) {
FILE *in_file = fopen(argv[1], "r");
char data[255];
fscanf(in_file, "%[^\n]", data);
P4_Grammar* grammar = P4_CreateGrammar();
if (grammar == NULL) {
return 1;
}
if (P4_AddLiteral(grammar, ENTRY, data, false) != P4_Ok) {
return 1;
}
P4_Source* source = P4_CreateSource(data, ENTRY);
if (source == NULL) {
return 1;
}
if (P4_Parse(grammar, source) != P4_Ok) {
return 1;
}
P4_Token* root = P4_GetSourceAst(source);
char* text = P4_CopyTokenString(root);
free(text);
P4_DeleteSource(source);
P4_DeleteGrammar(grammar);
return 1;
}
Ubuntu 18.04.5 LTS
Linux ubuntu 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Some special characters can't be parsed and it caused memory (heap-bufffer-overflow) errors.
You can produce it with string "Hello Worìd" (ì = 0xEC or 236). I patched P4_CaseCmpInsensitive function to show what is the value of P4_String given to P4_ReadRune as an argument. It can be seen that it is stopped after ì because it is parsing variable wrong and assign wrong number to P4_Rune* c . With the use of wrong value, program will access memory areas where it shouldn’t be. I am aware of open issue (#38) for supporting other utf encodings.
If input is a long string which contains “illegal” characters, program will be aboreted and throws DEADLYSIGNAL. This bug can be produced without sanitizers. Inpus can be found here: https://controlc.com/74f1e9b9
AFL++ (https://github.com/AFLplusplus/AFLplusplus) is used for this fuzzing campaign.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.