• Switch off of bulleted lists for the API documentation and use headers
• Break out Lisp example into annotated example in the examples folder
• Break out the custom parser into an annotated example in the examples folder

There are both minified and unminified browser files shipped with Parsimmon, so let's mention both in the README.

Right now we only can create parsers by combining existing ones. It is useful in some cases to allow for the creation of a base parser as well. The only thing missing for that right now is having access to makeSuccess and makeFailure.

In my example (https://github.com/khepin/parsimmon/blob/own_parsers/examples/jscomments.js), I setup a sample parser for JS comments trying not to use regex at all. For this I would need to be able inside a comment to match anything but the * character and there is no base parser for doing not('*')

If I build it through a regex parser: regex(/[^*]/) it works, but working on a 2 million comments file, the added time is quite important (8 seconds with regex vs 5 with a simple equality check on my machine and on average).

I only changed those lines: https://github.com/khepin/parsimmon/blob/own_parsers/src/parsimmon.js#L67 but I somehow feel this is not matching the overall style of this lib. Another option would be to pass success and failure arguments to the _() function like: _(stream, i, success, failure) but that seems a bit too troublesome and error prone to use afterwards.

What are your thoughts on this?

second condition in custom parser (notChar) should be
if (stream.charAt(i) !== char && i <= stream.length) {
not
if (stream.charAt(i) !== char && stream.length <= i) {

I've used parsimmon for a bunch of projects and it works very nice, but I miss a build-in feature for separated lists.

It is easy to rig using many(), skip() and map(), but a general (optimal) solution could be nice.

Something like Parsimmon.sep(words, space) that would return an array of words with spaces.

I just throw it out there, see what you think.

In document, Parsimmon.fail(message) isn't described while document of Parsimmon.succeed(result) is written.

I'm not entirely sure how this would work out in JS-land (what with the lack of currying), but it's always been useful to wrap up parsers in, say, parens:

var between = Parsimmon.between;
var string = Parsimmon.string;
var parens = between(string('('), string(')');

var hiParens = parens(string('hi'));

hiParens.parse('(hi)').value // => 'hi'

Thoughts?

Hello,

I just tried Parsimmon and somehow "or" does not always what I believe it should do. Here is a small example

var plus = Parsimmon.string("+")
var word = Parsimmon.string("a")
var operation = Parsimmon.seq(word, plus)
var term = word.or(operation)

console.log(operation.parse("a+")) // status: true
console.log(word.parse("a+"))  // status: false
console.log(term.parse("a+")) // status: false

The operation can parse "a+" but word.or(operation) cannot. The or function works with simple strings. I think it does not work due to the seq in the or statement. The expression operation.or(word) works fine, but I need it in this order for my complex example. Am I using it wrong?

Now, parse method is received one target string.

However, I want to give with offset, such as:

var result = parser.parse("target string!", 7);

It seems not difficult to change, and is compatible. How's this?

when working with a parser object in a terminal I expected parser.constructor.name to output something useful like Parser instead it's C because P creates a wrapper function using C

Are there any plans to add support for look ahead combinators? I think we need a generic way of "trying", that is having a look-ahead combinator, and then implement other combinators such as "notFollowedBy" on top of it.

Is it possible to write a parser that matches a recursive grammar? such as:

declaration-list: declaration [ ';' S* declaration-list ]?
                | at-rule declaration-list
                | /* empty */;

Hey!
The doc says P.alt is backtracking, however it doesn't appear to be the case here.
To be clear: the way I understand 'backtracking' is that if the first parser fails on input, the next parser will be tried on the same input.
I am not a 100% certain I understand the docs correctly so just wanted to check if this is the correct understanding before giving you repro steps and/or a PR fixing it (I have a pretty big grammar so it will take me some time to give nice repro steps...).

Went to use sepBy and noticed it wasn't in the current npm release. :)

I often want a optional parser. Now I use .times(0, 1) for this. However, .times(0, 1) doesn't seem to mean optional. When you read code and find .times(0, 1), can you see that .times(0, 1) creates optional parser?

I suggest .optional() method that is equal to .times(0, 1).

I was impressed by parsimmon. I want to let many people to know Parsimmon. So I have translated README.md of parsimmon into Japanese.
If you don't want me to publish this translated README.md, I don't publish. Can I write article about README.md of parsimmon in Japanese?

Hey there - first I want to say thanks for the great open source library.

I was wondering if we could get a few examples of different use cases. I'm having a hard time diving in an figuring out exactly what I need. I noticed the initial example on the README using a recursive solution with lazy returning form.or(atom). Later in the tips section you mention that generally lazy returns a call to alt.

I'm sure in the end these are trivial differences, but being new to the idea of combinator parsers, I'd just love to see some simple examples. (btw - I did notice the nice JSON parsing example, but it was a little much to get my head around). Hopefully it's not too much of a hassle - anything would be super helpful.

The API documentation is large enough that it clogs up the README.md. Let's move it to API.md.

I could not find a description of these methods in the readme.

Currently the regexp parser returns the whole match, while it would be very convenient if we could specify a match group.

Good for optional stuff in the RegExp: so if you have like /v?(\d+)/ you could get just the digits and ignore the 'v' prefix.

I have this working here. (looks funny because it optimises a conditional). Got some tests too.

I can PR this but since you guys are working on parsimmon I'd send a ticket first as I worked on the old error-throwing code.

[_] Assert that P.seqMap has at least 1 argument
[_] Assert the final argument is a function
[x] Assert every other argument is a parser- Already does this.

1. Add a sensible .npmignore
2. Add prepublish hook to build browser code
3. Add Mocha runner / file size reporter
4. Switch code to UMD so we don't need any source code transforms besides Uglify

When a parse fails and you check the .expected field, the array contains many non-unique elements. I would expect this list to be unique.

Contrived example

P.string("a").or(P.string("a")).parse("b")
// => { status: false, index: 0, expected: [ '\'a\'', '\'a\'' ] }

Real world example:

I can probably rephrase my grammar to avoid this somehow, but it seems less than ideal.

{ status: false,
  index: 20,
  expected:
   [ 'EOF',
     '\'#\'',
     'whitespace',
     '\'|>\'',
     '\'#\'',
     'whitespace',
     '\'or\'',
     '\'#\'',
     'whitespace',
     '\'and\'',
     '\'#\'',
     'whitespace',
     '\'!=\'',
     '\'#\'',
     'whitespace',
     '\'=\'',
     '\'#\'',
     'whitespace',
     '\'<=\'',
     '\'#\'',
     'whitespace',
     '\'<\'',
     '\'#\'',
     'whitespace',
     '\'>=\'',
     '\'#\'',
     'whitespace',
     '\'>\'',
     '\'#\'',
     'whitespace',
     '\'~\'',
     '\'#\'',
     'whitespace',
     '\'++\'',
     '\'#\'',
     'whitespace',
     '\'-\'',
     '\'#\'',
     'whitespace',
     '\'+\'',
     '\'#\'',
     'whitespace',
     '\'/\'',
     '\'#\'',
     'whitespace',
     '\'*\'',
     '\'#\'',
     'whitespace',
     '\'[\'',
     '\'.\'',
     '\'#\'',
     'whitespace' ] }

Hi.

Some methods use assertParser, but other methods, such as .seq(), .or() and .next() don't use it. I think they have to call assertParser() because they accept Parser.

assertParser isn't necessary? Or some methods should use it?

I execute npm install parsimmon, but I cannot use Parsimmon.sepBy. var parsimmon = require('parsimmon'); doesn't contain .sepBy() and .sepBy1(). I guess that parsimmon isn't built for npm now. I want it to be built and available on npm. Please build it.

Hello,

Thank you for this parser library. I have a lot of fun playing around with it! :)

After cloning the project I noticed that it doesn't build on my machine, because the node modules 'mocha' and 'chai' were missing:

sven@muckelmaus ~/s/parsimmon> make test
./node_modules/.bin/mocha -u tdd ./test/intro.js ./test/*.test.js
/bin/sh: 1: ./node_modules/.bin/mocha: not found
Makefile:38: recipe for target 'test' failed
make: *** [test] Error 127
sven@muckelmaus ~/s/parsimmon> npm install mocha --save-dev
npm WARN package.json [email protected] No license field.
[email protected] node_modules/mocha
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected])
sven@muckelmaus ~/s/parsimmon> make test
./node_modules/.bin/mocha -u tdd ./test/intro.js ./test/*.test.js
module.js:340
    throw err;
    ^

Error: Cannot find module 'chai'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:289:25)
    at Module.require (module.js:366:17)
    at require (module.js:385:17)
    at Object.<anonymous> (/home/sven/src/parsimmon/test/intro.js:1:79)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Module.require (module.js:366:17)
    at require (module.js:385:17)
    at /home/sven/src/parsimmon/node_modules/mocha/lib/mocha.js:216:27
    at Array.forEach (native)
    at Mocha.loadFiles (/home/sven/src/parsimmon/node_modules/mocha/lib/mocha.js:213:14)
    at Mocha.run (/home/sven/src/parsimmon/node_modules/mocha/lib/mocha.js:453:10)
    at Object.<anonymous> (/home/sven/src/parsimmon/node_modules/mocha/bin/_mocha:393:18)
    at Module._compile (module.js:435:26)
    at Object.Module._extensions..js (module.js:442:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:313:12)
    at Function.Module.runMain (module.js:467:10)
    at startup (node.js:136:18)
    at node.js:963:3
Makefile:38: recipe for target 'test' failed
make: *** [test] Error 1
sven@muckelmaus ~/s/parsimmon> npm install chai --save-dev
npm WARN package.json [email protected] No license field.
[email protected] node_modules/chai
├── [email protected]
├── [email protected]
└── [email protected] ([email protected])

npm install mocha --save-dev and npm install chai --save-dev did the trick. There are now two additional entries in package.json:

  "devDependencies": {
    "chai": "^3.4.1",
    "mocha": "^2.3.4",
...
  },

Should I file a pull request for this (very small) change? It might be that the version numbers don't fit to the project as there is still one failing test:

sven@muckelmaus ~/s/parsimmon> make test
./node_modules/.bin/mocha -u tdd ./test/intro.js ./test/*.test.js


  parser
    ✓ Parsimmon.string
    ✓ Parsimmon.regex
    ✓ Parsimmon.regex with group
    1) Parsimmon.seq
    ✓ Parsimmon.alt
    ✓ eof
    ✓ test
    ✓ takeWhile
    ✓ index
    ✓ mark
    Parsimmon.custom
      ✓ simple parser definition
      ✓ failing parser
      ✓ composes with existing parsers
    Parsimmon.sepBy/sepBy1
      ✓ successful, returns an array of parsed elements
      ✓ sepBy succeeds with the empty list on empty input, sepBy1 fails
      ✓ does not tolerate trailing separators
    then
      ✓ with a parser, uses the last return value
    chain
      ✓ asserts that a parser is returned
      ✓ with a function that returns a parser, continues with that parser
    map
      ✓ with a function, pipes the value in and uses that return value
    result
      ✓ returns a constant result
    skip
      ✓ uses the previous return value
    or
      ✓ two parsers
      ✓ with chain
    many
      ✓ simple case
      ✓ followed by then
    times
      ✓ zero case
      ✓ nonzero case
      ✓ with a min and max
      ✓ atMost
      ✓ atLeast
    fail
      ✓ use Parsimmon.fail to fail dynamically
      ✓ use Parsimmon.succeed or Parsimmon.fail to branch conditionally
    smart error messages
      or
        ✓ prefer longest branch
        ✓ prefer last of equal length branches
        ✓ prefer longest branch even after a success
      many
        ✓ prefer longest branch even in a .many()
        ✓ prefer longest branch in .or() nested in .many()
      times
        ✓ prefer longest branch in .times() too
      desc
        ✓ allows custom error messages
        ✓ allows tagging with `lazy`


  40 passing (93ms)
  1 failing

  1) parser Parsimmon.seq:

      AssertionError: expected [ Array(3) ] to deeply equal [ '(', 'string between parens', ')' ]
      + expected - actual

       [
         "("
      -  [
      -    "s"
      -    "t"
      -    "r"
      -    "i"
      -    "n"
      -    "g"
      -    " "
      -    "b"
      -    "e"
      -    "t"
      -    "w"
      -    "e"
      -    "e"
      -    "n"
      -    " "
      -    "p"
      -    "a"
      -    "r"
      -    "e"
      -    "n"
      -    "s"
      -  ]
      +  "string between parens"
         ")"
       ]

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:207:32)
      at Context.<anonymous> (test/parsimmon.test.js:62:14)



Makefile:38: recipe for target 'test' failed
make: *** [test] Error 1

Kind regards,

Sven

I've been working on the ParseError as per #19, but have a interesting problem:

Some code:

https://github.com/Bartvds/parsimmon/blob/7cd7d2833ecac312721eaaced47bb04b4c055c15/src/parsimmon.js#L14-L26

Thing is: currently the parsers can return a quoted string as expectation for a string parse (eg: 'abc', or a non-quoted (vanilla) string for 'symbols' like eof etc. This makes it harder to handle the error and make a nice report (doable but a little messy).

I'm not sure how to put that distinction between parse strings and 'symbols' in the expected value of the error.

Any suggestions?

Parsimmon.seq() produces a mystifying error message if you pass it varargs rather than an array:

> var Parsimmon = require('parsimmon')
> Parsimmon.seq(Parsimmon.string("hi")).parse("hi")
Parse Error: expected EOF at character 0, got 'hi'
    parsing: 'hi'
> Parsimmon.seq([Parsimmon.string("hi")]).parse("hi")
[ 'hi' ]
> 

It would be nice if this checked the expected type at construction time, or better yet, accepted varargs rather than an array. All of the users in both my code and parsimmon itself take a fixed number of arguments.

The regex parser is currently fairly limited. In particular, it's difficult to write lexemes that need to be greedy because it can only match one group at a time:

> p.regex(/[+\-]?([0-9]+|[0-9]*\.[0-9]+)(e[+-]?[0-9]+)?/g).parse('+1.4')
{ status: false, index: 2, expected: [ 'EOF' ] }
> '+1.4'.match(/[+\-]?([0-9]+|[0-9]*\.[0-9]+)(e[+-]?[0-9]+)?/g)
[ '+1', '.4' ]

In the above example, it would be nice if there was an option to just get all of the matching groups and concatenate the result so it returns '+1.4'. I don't think there is a way to write the above lexer without splitting up the regex into multiple sections.

// parsimmon.js
...
        var fullMatch = match[0];
...

It's called out that P.alt returns the FIRST match, but it's worth emphasizing that this means ORDER MATTERS and you have to put longer parsers before shorter parsers.

Infix operators are hard, so let's make a calculator parser example.

Should support + - * / ^ and unary negation, along with parentheses for grouping.

There's no need for that. Just use RegExp.prototype.exec and make sure the index property of the result is 0. I'll send over a PR soon.

In fantasyland/fantasy-land#55, you mentioned that this might be a Monoid with or and fail. This would of course mean that or would have to be the Semigroup operation. However, I'm not sure this is the ONLY Semigroup/Monoid that you have in here. I think you have at least two.

• A choice Semigroup
• A combination Semigroup

What I mean by that is you can think of parsing a string in at least two ways.

• Parse with this parser, if that fails, parse with this other parser.
• Parse with this parser, and also parse with this other parser.

respectively.

I'm not sure which one you view as more natural, but it might be helpful to have at least one more concat method. If you see the combination Semigroup as the default behavior, then concatOr which gives you the choice semantics. If you see the choice Semigroup as the default behavior, then concatAnd gives you the combination behavior. Or if you want to provide boh, that makes sense too, just default concat to one of those. Of course, the naming is up to you.

You've already got the choice Semigroup being a Monoid. To make the combination Semigroup into a Monoid, you just need a parser that always succeeds.

I'd tried installing parsimmon from npm on a windows workstation but installation bails on the pjs dependency:

The pjs module has a npm install script that runs make, which obviously doesn't exist on vanilla windows.

Ticket here: jneen/pjs#28

var sq = string('a').atMost(1);
console.log( sq.parse( 'a' ) );

this fails. what gives?

Some flags for regexps cannot be reasonably supported within Parsimmon, so let's make it an error to use them!

Only i, u, and m make sense to support https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp.

Thanks to everyone in #81 for the discussion that brought up this issue!

I think it might make sense to just put this check in the assertRegexp function.

Array merge can be optimized.

I'll probably rename the function to like unsafeUnion or something, since it assumes the other argument is sorted already when one array is size 0.

PEG.js and Jison give locations not only as a character offset but also as line and column numbers. These would help make errors human-readable and are necessary for various other tools e.g. source maps.

Calculating line and column positions of everything after parsing has finished mixes concerns and is inefficient.

Could parsimmon have e.g. an option passed to .parse that would calculate and attach line and column numbers to the objects produced by a .mark? Or maybe a new .markDescriptive could do that itself? (I'm speculating because I don't yet understand how they work internally.)

A quick search says something like this was discussed in 2014 #23 (comment), but nothing seems to have stuck.

The build is failing with this message:

npm ERR! Error: No compatible version found: camelcase@'^1.0.2'

This happened on the Adding sepBy and sepBy1 commit, and it appears to have resulted in those functions not being available in the npm package.

So with parsimmon I can do this (from the readme):

expr.parse('(add (mul 10 (add 3 4)) (add 7 8))').value
// => ['add', ['mul', 10, ['add', 3, 4]], ['add', 7, 8]]

How do I perform the opposite operation?

compose(['add', ['mul', 10, ['add', 3, 4]], ['add', 7, 8]])
// => "(add (mul 10 (add 3 4)) (add 7 8))"

Hello! I think there might be some issue with Parsimmon.alt's backtracing.

My understanding is that if Parsimmon.alt(a, b) succeeds, then Parsimmon.alt(b, a) should succeed, but that seems not to be that case.
https://tonicdev.com/575832c9c91caa1300f7bad6/577e7e356d5f74130004bae8 shows a couple of cases where things behave surprisingly. I would expect all of the test cases there to parse successfully.

Thank you for your awesome work maintaining this :)

If you call .parse with a non-string argument, the error message isn't very helpful. I've bumped into this a couple times refactoring my code.

Current error message

> P.string("x").parse(3)
TypeError: stream.slice is not a function
    at Parser._ (/Users/brian/squiggle/node_modules/parsimmon/build/parsimmon.commonjs.js:306:25)
    at Parser._ (/Users/brian/squiggle/node_modules/parsimmon/build/parsimmon.commonjs.js:113:42)
    at Parser._ (/Users/brian/squiggle/node_modules/parsimmon/build/parsimmon.commonjs.js:275:25)
    at Parser.Parsimmon.Parser._.parse (/Users/brian/squiggle/node_modules/parsimmon/build/parsimmon.commonjs.js:84:33)
    at repl:1:15
    at REPLServer.defaultEval (repl.js:164:27)
    at bound (domain.js:250:14)
    at REPLServer.runBound [as eval] (domain.js:263:12)
    at REPLServer.<anonymous> (repl.js:393:12)
    at emitOne (events.js:82:20)

Proposed error message

> P.string("foo").parse(2)
Error: .parse must be called with a string as its argument
    at Parser.Parsimmon.Parser._.parse (/Users/brian/parsimmon/build/parsimmon.commonjs.js:85:13)
    at repl:1:17
    at REPLServer.defaultEval (repl.js:164:27)
    at bound (domain.js:250:14)
    at REPLServer.runBound [as eval] (domain.js:263:12)
    at REPLServer.<anonymous> (repl.js:393:12)
    at emitOne (events.js:82:20)
    at REPLServer.emit (events.js:169:7)
    at REPLServer.Interface._onLine (readline.js:210:10)
    at REPLServer.Interface._line (readline.js:549:8)

I have a PR for this nearly ready.

Given that JS calls them RegExp, I think there should be a P.regexp function, and P.regex can just become a deprecated alias for it.

I have some feedback for the errors:

Main thing: currently the parser throws a formatted string instead of a real Error. This is a little limiting in the amount of info that can be recovered for reporting:

1. I would like to do some work with the reported character position, like calculating a line-number. This would be properties on the Error object.

2. The Regular Expression is printed in the message.

When using a RegExp to match all Unicode 'letter' (using XRegExp's \p{L} category) this can happen (LoL :)

This is expected (and fine) for development but I'd like make it more sane for our users (eg: the developers who are contributing the text we're parsing here).

This can be dealt with easily if the result would be an object with some fields.

3. I would like to be able to 'name' and/or 'describe' parsers and use that name in my applications report.

Maybe to keep current API clean we could add a new method that would return a new, decorated parser:

// simple
parser = parser.describe('name', 'some human label');

// creates new parser so this would be safe
fooA = P.digits.describe('fooA');
fooB = P.digits.describe('fooB');

// gold plated idea with some predefined variables
parser = parser.describe('name', 'some human label', 'custom error string with vars {{line}}, {{row}}'));

I'm forking and having a go at these myself (I spin up my Linux gear) but I would value your opinion and wonder if you'd consider these kind of expansions for merging back later.

(btw: I'm glad I pushed for the window fix earlier: parsimmon proves to be really nice to work with)

Output of install:

$ npm install parsimmon
npm WARN package.json [email protected] No readme data.
npm http GET https://registry.npmjs.org/parsimmon
npm http 200 https://registry.npmjs.org/parsimmon
npm http GET https://registry.npmjs.org/parsimmon/-/parsimmon-0.2.0.tgz
npm http 200 https://registry.npmjs.org/parsimmon/-/parsimmon-0.2.0.tgz
npm http GET https://registry.npmjs.org/pjs
npm http 200 https://registry.npmjs.org/pjs
npm http GET https://registry.npmjs.org/pjs/-/pjs-3.1.0.tgz
npm http 200 https://registry.npmjs.org/pjs/-/pjs-3.1.0.tgz

> [email protected] install C:\Users\micah\Dropbox\printmee\radom-apps\margot\node_modul
es\parsimmon\node_modules\pjs
> make commonjs

'make' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! weird error 1
npm ERR! not ok code 0

I was trying to use the .desc() method as described on the Github README but it looks like it is not yet published on npm.

Could you push a new version with this?

I noticed there is no licence noted in the readme or package.json, nor is there a Licence file in the repos.

Could you see if this can be fixed? Without it the module is not really usable (and I'm maybe already in copyright breach for creating these.

A lot of changes has been pushed to the repository since the latest release.
Can you bump the NPM version?

This library looks very interesting, thanks for it!

I have a question, how is the resulting parser CPU and memory efficient in comparison with other JS parsers? It would be very useful to provide some benchmarks for performance comparison.