itod / parsekit Goto Github PK

I'm trying to play around with Parsekit but the DemoApp throws an exception when I press "Parse" with the default text, both on the Parse Tree and AST tabs.

this parsekit it's not compatible with the armv7s architecture of the new iphone 5, anyone know how add it?

http://parsekit.com/ still links to "Google Code", which hasn't been updated in years, literally. (The "Source Code" link in the top left corner, just to begin with)

In my grammar, I try to add support of whitespace ...

combinator              = '>' | '~' | plus | S;

The grammar generate a parser, but the generated parser compiles with error:

Use of undeclared identifier 'TOKEN_KIND_BUILTIN_S'; did you mean 'TOKEN_KIND_BUILTIN_EOF'?
No visible @interface for 'CSSSelectorParser' declares the selector 'matchS:'

I noticed the whitespace token is TOKEN_KIND_BUILTIN_WHITESPACE and matchWhitespace:, so I try to write grammar like this:

combinator              = '>' | '~' | plus | Whitespace;

This is not a valid grammar however. Can you advice how I can properly use whitespace? Thanks a lot.

Trying to generate a parser for the MiniMath grammar from your tutorial, it throws an exception when trying to generate for the provided grammar. I also tried rolling back to the previous commit (the same one which is pointed to by the MiniMath repo submodule pointer) and got the same result.

I also made sure to make the ParserGenApp options match what was shown in the tutorial.

When directly cloning the MiniMath project (and its parsekit submodule), it works as expected.

I haven't been able to include and compile the 0.7 podspec, I keep getting missing symbols.
And looking at files including in xcode, there's no .m files

HI, I just downloaded and tried the iPhone instructions with XCode 4. They didn't work and just brought in the poject file instead of the project.

So I switched to a set of build scripts I have which build iPhone/iPad universal static frameworks. I used the the build script below and my scripts at https://github.com/drekka/dUsefulStuff/tree/master/scripts along with appledoc. The scripts perform builds for arm and i386 hardware, build documentation and assemble distributable (versioned - I used v1.0.0 because I could not a version on the parsekit code) dmg files.

I thought you might find these scripts interesting. I build a number of static frameworks with them now, both my own and cloned OSS projects.

Here's the main build script I used to drive the build:

#!/bin/sh

# build.sh
#
# Created by Derek Clarkson on 27/08/10.
# Copyright 2010 Derek Clarkson. All rights reserved.

# build specific.
DC_CURRENT_PROJECT_VERSION=${CURRENT_PROJECT_VERSION=1.0.0}
DC_PRODUCT_NAME=${PRODUCT_NAME=ParseKit}
DC_LIB_PREFIX=
DC_SRC=include/ParseKit
DC_BUILD_TARGET="ParseKitMobile"
DC_COMPANY_ID=com.parsekit.ParseKit
DC_AUTHOR="Todd Ditchendorf"
DC_COMPANY=$DC_AUTHOR

DC_SIMULATOR_SDK=iphonesimulator4.3
DC_DEVICE_SDK=iphoneos4.3

DC_SCRIPTS_DIR=../../dUsefulStuff/scripts

# Include common scripts.
source $DC_SCRIPTS_DIR/defaults.sh
source $DC_SCRIPTS_DIR/common.sh

# Clean and setup.
$DC_SCRIPTS_DIR/clean.sh

# Check for a doco only build.
if [ -n "$DC_BUILD_DOCO_ONLY" ]; then
    $DC_SCRIPTS_DIR/createDocumentation.sh
    exit 0
fi

# Otherwise do a full build.
$DC_SCRIPTS_DIR/buildStaticLibrary.sh
$DC_SCRIPTS_DIR/assembleFramework.sh
$DC_SCRIPTS_DIR/createDocumentation.sh

# Final assembly.
$DC_SCRIPTS_DIR/createDmg.sh

I'm trying to use your PKTreeAssembler. Yes, I know it is in the test package, not in the framework but I still need it. It is sufficiently complex so I refrain from writing one myself. I have it working (after few changes for ARC) but the returned PKParseTree contains only rule nodes and no tokens at all. I have tried to debug it - it seems OK. It seems to add PKToken nodes to the tree. Somehow they disappear from the returned tree. What could be wrong? Thanks.

/Mikael

The intro documentation is great and got me started. But I need more details. The HeaderDoc links are all dead and from the source code it appears that they would not be much help anyway. What I'm looking for:

1. Reference for the grammer syntax. Most specifically keywords. Just referring to BNF is not enough because parsekit's syntax appears to be different and doesn't give us the built ins. For example "Word" appears to be a built in keyword.
2. How does parsekit decide what methods to call on the call back. From the examples it appear they these are dynamically produced, but the doco doesn't tell me how and it appears that parsekit doesn't either. So I'm guessing at what methods I have to include.

2013-11-09 02:26:41.185 ParserGenApp[82193:5f0b] missing rule named: `functionDef`
2013-11-09 02:26:41.185 ParserGenApp[82193:5f0b] *** Assertion failure in -[PKSParserGenVisitor lookaheadSetForNode:], /Users/fbarthelemy/Code/parsekit/src/PKSParserGenVisitor.m:163
[snipped]

1. Run the ParserGenApp using the obj.grammar file.
2. Notice how it crashes with error missing rule named 'functionDef'
   Expected behavior: a parser would be generated.

Diff to Repro:

diff --git a/ParseKit.xcodeproj/project.pbxproj b/ParseKit.xcodeproj/project.pbxproj
index ce77a6e..4442ae0 100644
--- a/ParseKit.xcodeproj/project.pbxproj
+++ b/ParseKit.xcodeproj/project.pbxproj
@@ -7,6 +7,7 @@
    objects = {

 /* Begin PBXBuildFile section */
+       2E3956D0182E4518003E769B /* objc.grammar in Resources */ = {isa = PBXBuildFile; fileRef = D353899D0FEAC96B00926552 /* objc.grammar */; };
        D302C69D0EF044810090E714 /* TDPlistParser.m in Sources */ = {isa = PBXBuildFile; fileRef = D36568A60EEF9FE900226554 /* TDPlistParser.m */; };
        D30805BA155ECB1F0000BC60 /* PKPattern.m in Sources */ = {isa = PBXBuildFile; fileRef = D34BAF380FF9E18300D7773A /* PKPattern.m */; };
        D30805BB155ECB220000BC60 /* RegexKitLite.m in Sources */ = {isa = PBXBuildFile; fileRef = D36C55FD0FD3617B00141CB7 /* RegexKitLite.m */; };
@@ -3705,6 +3706,7 @@
                D33830A0171CA3BB00CCE513 /* PKSPredictElseIfTemplate.txt in Resources */,
                D33830A1171CA3BB00CCE513 /* PKSSpeculateElseIfTemplate.txt in Resources */,
                D33830A2171CA3BB00CCE513 /* PKSPredictElseTemplate.txt in Resources */,
+               2E3956D0182E4518003E769B /* objc.grammar in Resources */,
                D33830A3171CA3BB00CCE513 /* PKSPredictEndIfTemplate.txt in Resources */,
                D33830A4171CA3BB00CCE513 /* PKSOptionalSpeculateTemplate.txt in Resources */,
                D33830A5171CA3BB00CCE513 /* PKSOptionalPredictTemplate.txt in Resources */,
diff --git a/ParserGenApp/PGDocument.m b/ParserGenApp/PGDocument.m
index 2f9f99b..67fbe10 100644
--- a/ParserGenApp/PGDocument.m
+++ b/ParserGenApp/PGDocument.m
@@ -35,7 +35,7 @@
         self.preassemblerSettingBehavior = PKParserFactoryAssemblerSettingBehaviorNone;
         self.assemblerSettingBehavior = PKParserFactoryAssemblerSettingBehaviorAll;

-        NSString *path = [[NSBundle mainBundle] pathForResource:@"expression" ofType:@"grammar"];
+        NSString *path = [[NSBundle mainBundle] pathForResource:@"objc" ofType:@"grammar"];
         self.grammar = [NSString stringWithContentsOfFile:path encoding:NSUTF8StringEncoding error:nil];
     }
     return self;

DemoApp keeps crashing on me whenever I happen to have entered an invalid grammar.
I might have got the purpose of DemoApp all wrong, but in my eyes this kinda renders DemoApp pretty useless, doen't it?

To reproduce those crashes simply either:

• randomly remove semicolons from DemoApp's default grammar.
  or
• change one of the keywords from DemoApp's default grammar to something not defined.
  and hit "Parse". Boom. Crash.

I've just discovered that a rather odd error I was getting when compiling my code was because parsekit uses CGFloat in PKToken. CGFloat comes from CoreGraphics means that CoreGraphics/CGBase.h has to be included before PKToken.h. The error generated is this:

In file included from /Users/derek/projects/Simon/classes/SIStoryReader.m:12:
/Users/derek/projects/Simon/../Releases/ParseKit/v1.0.0/ParseKit.framework/Headers/PKToken.h:50: error: expected specifier-qualifier-list before 'CGFloat'
/Users/derek/projects/Simon/../Releases/ParseKit/v1.0.0/ParseKit.framework/Headers/PKToken.h:82: error: expected ')' before 'CGFloat'
/Users/derek/projects/Simon/../Releases/ParseKit/v1.0.0/ParseKit.framework/Headers/PKToken.h:91: error: expected ')' before 'CGFloat'
/Users/derek/projects/Simon/../Releases/ParseKit/v1.0.0/ParseKit.framework/Headers/PKToken.h:177: error: expected specifier-qualifier-list before 'CGFloat'

Doing this fixes it:

#import <CoreGraphics/CGBase.h>
#import <ParseKit/PKToken.h>

Can the type be changed to something else as it enforces a dependency on CoreGraphics which may not be relevant.

2012-01-26 12:34:27.862 DebugApp[74078:507] -[TDGenericAssembler parser:didMatchOpenCurly:]: unrecognized selector sent to instance 0x105e2db80

when calling - (void)doJSONHighlighting

This version of XCode uses directory 'DerivedData' to keep all generated files on. Therefore on my local clone of your repository I added 'DerivedData' to .gitignore and then removed it from the repository. I think you may want to know. Thanks.

Mikael Hakman

It seems the interface of NSRange has changed, and so line 84 of TDSourceCodeTextView.m which reads:

NSUInteger length = index + [visibleGlyphRange length];

needs to be changed to:

NSUInteger length = index + visibleGlyphRange.length;

in order to compile properly.

Will you create a version of parsekit that support ARC?

Thanks for this great lib.

What steps will reproduce the problem?

1. Run DemoApp
2. type a "w" in the string to tokenize on the Tokens tab
3. click the Parse button

What is the expected output? What do you see instead?
There should be a single token "w". Instead, you get a single token "ww".

What version of the product are you using? On what operating system?
I used the latest code from git repository.

Please provide any additional information below.

I tracked this down to the methods in PKURLState.m
A: - (BOOL)parseWWWFromReader:(PKReader *)r
B: - (PKToken *)nextTokenFromReader:(PKReader *)r startingWith:(PKUniChar)cin tokenizer:(PKTokenizer *)t

1. - B calls A to attempt to find a URL starting with www.
2. - A does a read to look for more 'w' characters.
3. - control returns to B and matched is NO so B does an unread to undo what A potentially read.
4. - parsing continues

The problem is when the 'w' is at the end of the input string. Step 2 reads nothing so step 3 duplicates the w with the unread. If you type in "ww" and press parse, you get "wwww" out. This is also true regardless of what is before this token. "i am going to go talk to my w" will come out with tokens "i", "am", "going", "to", "talk", "to", "my", "ww".

Solution
I am thinking that - (BOOL)parseWWWFromReader:(PKReader *)r should do it's own cleanup (unread if necessary) before returning instead of the cleanup happening outside the method.

Work-around
You can avoid this bug by setting the tokenizer's URLState.allowsWWWPrefix to NO.

RegexKitLite should be removed as a dependency and NSRegularExpression (docs) should be used in its place. NSRegularExpression has been supported in OSX since 10.7.0 and in iOS for much longer and would help reduce bloat and compile time because RegexKitLite is filled with macros and outdated C functions.

I believe it should be @rpath/ instead of @executable_path/../Frameworks/

• this says I am on the run path instead of I am explicitly at this location relative to the main executable
• I ran into this issue when building a quick look generator plugin. I need to be relative to my plugin (@loader_path) rather than the quick look daemon (@executable_path).

Then, any application or plugin or other bundle using in should set it's Runpath search paths to @loader_path/../Frameworks or wherever else it wants to search. This way, the ParseKit framework will work in both executables and plugins.

All assignment inside a statement should be enclosed in a double parenthesis to avoid Clang compiler warnings.

For example, the assignment should look like this:
if ((self = [super init])) { }

Instead of:
if (self = [super init]) { }

Running the following parser results in some random results. The result occasionally contains "search" and/or "for", assumedly because of the PKAny Repetition immediately before and after the PKAlternation. Is there any way to resolve this or am I stuck with it?

    NSString *string = @"google search for ducky ducky super ducky duck";
    PKSequence *parser = [PKSequence sequence];

    PKRepetition *repetitionTest = [PKRepetition repetitionWithSubparser:[[PKAny any] discard]];

    [parser add:repetitionTest];

    PKAlternation *optionalSearch = [PKAlternation alternation];
    [optionalSearch add:[[PKEmpty empty] discard]];
    [optionalSearch add:[[PKLiteral literalWithString:@"search"] discard]];
    [parser add:optionalSearch];

    [parser add:[[PKLiteral literalWithString:@"google"] discard]];
    [parser add:optionalSearch];

    PKAlternation *optionalFor = [PKAlternation alternation];
    [optionalFor add:[[PKEmpty empty] discard]];
    [optionalFor add:[[PKLiteral literalWithString:@"for"] discard]];

    [parser add:optionalFor];

    PKRepetition *searchTerm = [PKRepetition repetitionWithSubparser:[PKAny any]];
    [parser add:searchTerm];
    PKAssembly *result = [parser bestMatchFor:[PKTokenAssembly assemblyWithString:string]]

Now that OS X finally got native regex support it's time to think about making use of it instead of RegexKitLite, no?

Why does ParseKit depend on the CGFloat type?

This prevents a Fundation-only library to use ParseKit.

I’ve recently created a pull request to add ParseKit to the CocoaPods package manager repo.

CocoaPods is a tool for managing dependencies for OSX and iOS Xcode projects and provides a central repository for iOS/OSX libraries. This makes adding libraries to a project and updating them extremely easy and it will help users to resolve dependencies of the libraries they use.

However, ParseKit doesn't have any version tags. I’ve added the current HEAD as version 0.0.1, but a version tag will make dependency resolution much easier.

Semantic version tags (instead of plain commit hashes/revisions) allow for resolution of cross-dependencies.

In case you didn’t know this yet; you can tag the current HEAD as, for instance, version 1.0.0, like so:

$ git tag -a 1.0.0 -m "Tag release 1.0.0"
$ git push --tags