jgosmann / bite-parser Goto Github PK

View Code? Open in Web Editor NEW

6.0 4.0 1.0 3.25 MB

Asynchronous parser taking incremental bites out of your byte input stream.

Home Page: https://jgosmann.github.io/bite-parser/docs/en/stable/

License: MIT License

Python 100.00%

parser parsing parsing-expression-grammar peg parser-combinator bytes asyncio incremental python python3

bite-parser's Introduction

Welcome to bite-parser

Asynchronous parser taking incremental bites out of your byte input stream.

The bite-parser is a parser combinator library for Python. It is similar to PyParsing in that it allows the construction of grammars for parsing from simple building blocks in pure Python. This approach is also known as Parsing Expression Grammar (PEG). While PyParsing (and many other Python parsing libraries) only support string, bite-parser operates on bytes. In addition, bite-parser makes use of asyncio and can asynchronously generate parsed items from an input stream.

A typical use-case would be the parsing of a network protocol like IMAP. In fact, I wrote this library for the IMAP implementation of my dmarc-metrics-exporter.

Note

I have implemented the fundamental set of parsers, which should allow constructing most or all grammars recognizable by this type of parser. However, many convenience or higher level parsers are not yet implemented.

Other areas that still need improvement are:

Abilitiy to debug the parsing.
Better error messages.
Performance: Currently, only a basic recursive descent parser is implemented which can exhibit exponential worst case performance. This could be improved by implementing a packrat parser.

Important links

bite-parser's People

Contributors

Stargazers

Watchers

Forkers

dotlambda

bite-parser's Issues

When passing str where bytes is expected, fail fast or handle gracefully

Currently no error is produced at runtime when accidentally passing str instead of bytes, for example when using Literal('foo') instead of Literal(b'foo'), but parsing does not work. This should either raise an error or be handled gracefully (e.g. by encoding the string to bytes with ascii encoding).

Improve errors

Produce more helpful errors.
E.g. show which variant was tried to be parsed and failed.

Location marker in UnmetExpectationError marks wrong position

When the buffer is a bytearray (instead of bytes) the wrong location will be marked when an UnmetExpectationError is printed.

Come up with consistent naming strategy

The naming of parse nodes (from the parsers) is somewhat inconsistent and random at the moment. This should be made consistent. Also, names should not try to represent the sub-parse-tree, but make it easy to find specific nodes. Thus, it might be sufficient to just use the type of parser.

Add helper parsers/functions

Quoted strings (incl. quote removal)
parenthesized lists
nested expressions

Refactor start_loc/end_loc tracking

I think it would be a bit simpler to track a single extent (as 2-tuple) than two separate values in the ParseNodes.

Improve debug output

Enable useful debug output that clarifies why and how parsing is failing.

is it an issue ?

It seems i cannot do some elementary parsing

is it me ? or ?

#!/bin/env python

import asyncio
from bite import Parser, CharacterSet, Combine, Group, Literal, parse_bytes, Suppress, Opt, ZeroOrMore

upper_case= CharacterSet(b'ABCDEFGHIJKLMNOPQRSTUVWXZ')

lower_case= CharacterSet(b'abcdefghijklmnopqrstuvwxz')

upper_case= CharacterSet(bytes(range(ord(b'A'),1+ord(b'Z'))))
lower_case= CharacterSet(bytes(range(ord(b'a'),1+ord(b'z'))))

digit = CharacterSet(b'0123456789')
digits = digit[1, ...]

integer = Combine(digits)
letter=lower_case|upper_case
char=letter|digit

ident = Combine(letter + (char | '_')[0, ...])

obj = parse_bytes( ident, "un")
result = asyncio.run( obj )
print(result

Input: 'un'
^ location of error