macmade / clangkit Goto Github PK
View Code? Open in Web Editor NEWClangKit provides an Objective-C frontend to LibClang. Source tokenization, diagnostics and fix-its are actually implemented.
ClangKit provides an Objective-C frontend to LibClang. Source tokenization, diagnostics and fix-its are actually implemented.
A use case I have for ClangKit is, given a file, get the translation unit for it and then parse the stream of tokens into a tree-like structure. This requires getting a valid cursor for the tokens, so I can identify class and method declarations. Often, I just have a .m
or .mm
file, and calling [CKTranslationUnit translationUnitWithPath:somePath]
does the right thing.
However, if I call [CKTranslationUnit translationUnitWithPath:@"/my/special/file.h"]
then the actual tokens returned say the cursor is "NoDeclFound" for the entire file. You can see this below:
(lldb) po translationUnit.tokens
<__NSArrayI 0x6000001c0000>(
<CKToken: 0x600000085b40>: Punctuation[1:1] # (inclusion directive),
<CKToken: 0x600000085b90>: Identifier[1:2] import (inclusion directive),
<CKToken: 0x600000085be0>: Punctuation[1:9] < (inclusion directive),
<CKToken: 0x600000085c30>: Identifier[1:10] Foundation (NoDeclFound),
<CKToken: 0x600000085c80>: Punctuation[1:20] / (NoDeclFound),
<CKToken: 0x600000085cd0>: Identifier[1:21] Foundation (NoDeclFound),
<CKToken: 0x600000085d20>: Punctuation[1:31] . (NoDeclFound),
<CKToken: 0x600000085d70>: Identifier[1:32] h (NoDeclFound),
<CKToken: 0x600000085dc0>: Punctuation[1:33] > (NoDeclFound),
<CKToken: 0x600000085e10>: Punctuation[2:1] @ (NoDeclFound),
<CKToken: 0x600000085e60>: Identifier[2:2] interface (NoDeclFound),
<CKToken: 0x600000085eb0>: Identifier[2:12] XMASAlert (NoDeclFound),
<CKToken: 0x600000085f00>: Punctuation[2:22] : (NoDeclFound),
<CKToken: 0x600000085f50>: Identifier[2:24] NSObject (NoDeclFound),
<CKToken: 0x600000085fa0>: Punctuation[3:1] - (NoDeclFound),
<CKToken: 0x600000085ff0>: Punctuation[3:3] ( (NoDeclFound),
<CKToken: 0x600000086040>: Keyword[3:4] void (NoDeclFound),
<CKToken: 0x600000086090>: Punctuation[3:8] ) (NoDeclFound),
<CKToken: 0x6000000860e0>: Identifier[3:9] flashMessage (NoDeclFound),
<CKToken: 0x600000086130>: Punctuation[3:21] : (NoDeclFound),
<CKToken: 0x600000086180>: Punctuation[3:22] ( (NoDeclFound),
<CKToken: 0x6000000861d0>: Identifier[3:23] NSString (NoDeclFound),
<CKToken: 0x600000086220>: Punctuation[3:32] * (NoDeclFound),
<CKToken: 0x600000086270>: Punctuation[3:33] ) (NoDeclFound),
<CKToken: 0x6000000862c0>: Identifier[3:34] message (NoDeclFound),
<CKToken: 0x600000086310>: Punctuation[3:41] ; (NoDeclFound),
<CKToken: 0x600000086360>: Punctuation[4:1] @ (NoDeclFound),
<CKToken: 0x6000000863b0>: Identifier[4:2] end (NoDeclFound)
)
I have found a workaround of calling CKTranslationUnit translationUnitWithText: language:
, which actually does provide the correct cursor for the header. This looks a bit like this:
(lldb) po translationUnit.tokens
<__NSArrayI 0x6000001c12c0>(
<CKToken: 0x600000085910>: Punctuation[1:1] # (inclusion directive),
<CKToken: 0x6000000859b0>: Identifier[1:2] import (inclusion directive),
<CKToken: 0x600000085730>: Punctuation[1:9] < (inclusion directive),
<CKToken: 0x600000085690>: Identifier[1:10] Foundation (NoDeclFound),
<CKToken: 0x6000000856e0>: Punctuation[1:20] / (NoDeclFound),
<CKToken: 0x6000000855f0>: Identifier[1:21] Foundation (NoDeclFound),
<CKToken: 0x600000085640>: Punctuation[1:31] . (NoDeclFound),
<CKToken: 0x600000085550>: Identifier[1:32] h (NoDeclFound),
<CKToken: 0x600000085500>: Punctuation[1:33] > (NoDeclFound),
<CKToken: 0x6000000855a0>: Punctuation[2:1] @ (ObjCInterfaceDecl),
<CKToken: 0x6000000854b0>: Keyword[2:2] interface (ObjCInterfaceDecl),
<CKToken: 0x600000085460>: Identifier[2:12] XMASAlert (ObjCInterfaceDecl),
<CKToken: 0x600000085410>: Punctuation[2:22] : (ObjCInterfaceDecl),
<CKToken: 0x6000000853c0>: Identifier[2:24] NSObject (ObjCSuperClassRef),
<CKToken: 0x600000085320>: Punctuation[3:1] - (ObjCInstanceMethodDecl),
<CKToken: 0x600000085370>: Punctuation[3:3] ( (ObjCInstanceMethodDecl),
<CKToken: 0x600000085280>: Keyword[3:4] void (ObjCInstanceMethodDecl),
<CKToken: 0x600000085230>: Punctuation[3:8] ) (ObjCInstanceMethodDecl),
<CKToken: 0x6000000852d0>: Identifier[3:9] flashMessage (ObjCInstanceMethodDecl),
<CKToken: 0x6000000851e0>: Punctuation[3:21] : (ObjCInstanceMethodDecl),
<CKToken: 0x600000085190>: Punctuation[3:22] ( (ObjCInstanceMethodDecl),
<CKToken: 0x600000085140>: Identifier[3:23] NSString (ObjCClassRef),
<CKToken: 0x6000000850a0>: Punctuation[3:32] * (ParmDecl),
<CKToken: 0x6000000850f0>: Punctuation[3:33] ) (ParmDecl),
<CKToken: 0x600000085000>: Identifier[3:34] message (ParmDecl),
<CKToken: 0x600000085050>: Punctuation[3:41] ; (ObjCInstanceMethodDecl),
<CKToken: 0x600000084fb0>: Punctuation[4:1] @ (ObjCInterfaceDecl),
<CKToken: 0x600000084f60>: Keyword[4:2] end (ObjCInterfaceDecl)
)
I'd be more than happy to issue a pull request to fix this, but I'm not entirely sure where to start. This seems hard to fix because any arbitrary .h
file could be C, C++, Obj-C or even Obj-C++.
I'm getting a bunch of linker errors when compiling ClangKit.
I ran make in the LLVM directory. Is there anything else I need to do to get these problems solved?
By the way I'm on Xcode 5.1 and OS X 10.9.
Compressing objects: 100% (254/254), done. Writing objects: 100% (450/450), 386.98 MiB | 45 KiB/s, done. Total 450 (delta 192), reused 436 (delta 184) remote: Error code: ccfbbe687a6acf7a75bcd9a65554d6cd remote: warning: Error GH413: Large files detected. remote: warning: See http://git.io/iEPt8g for more information. remote: warning: File ClangKit/LLVM/Debug/lib/libclang.a is 59.75 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File ClangKit/LLVM/Debug/lib/libclangARCMigrate.a is 92.75 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File ClangKit/LLVM/Debug/lib/libclangStaticAnalyzerCore.a is 91.92 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: error: File ClangKit/LLVM/Debug/lib/libclangAST.a is 123.01 MB; this exceeds GitHub's file size limit of 100 MB remote: error: File ClangKit/LLVM/Debug/lib/libclangSema.a is 191.36 MB; this exceeds GitHub's file size limit of 100 MB remote: error: File ClangKit/LLVM/Debug/lib/libLLVMCodeGen.a is 101.46 MB; this exceeds GitHub's file size limit of 100 MB remote: error: File ClangKit/LLVM/Debug/lib/libclangCodeGen.a is 147.11 MB; this exceeds GitHub's file size limit of 100 MB remote: error: File ClangKit/LLVM/Debug/lib/libclangStaticAnalyzerCheckers.a is 210.42 MB; this exceeds GitHub's file size limit of 100 MB To https://github.com/mralexgray/ClangKit.git ! [remote rejected] master -> master (pre-receive hook declined)
Great library, hopefully I can get this to work with an iOS project, But, I'm having some issue . I've run make in the LLVM directory, according to the tutorial, then I open the included project to build the framework, but I get some errors such as
Undefined symbols for architecture x86_64:
"llvm::SpecialCaseList::create(llvm::StringRef, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >&)", referenced from:
clang::driver::SanitizerArgs::SanitizerArgs(clang::driver::ToolChain const&, llvm::opt::ArgList const&) in libclangDriver.a(SanitizerArgs.o)
I parse .m files written in Objective-C using ClangKit, when the source code is about over 300 lines, I find out that the source code is truncated and no tokens about the missing part, the snapshot bellow shows the problem.
If the source code is not too long, it works fine.
I have check out the source code, but can't find any problem.
I am very confused about this, can any one help? Thanks a lot.
When makefile run into line
'@$(CD)
it will output:
'The LLVM project no longer supports building with configure & make.
Please migrate to the CMake-based build system.
For more information see: http://llvm.org/docs/CMake.html'.
I guess maybe the version of 'ClangKit/LLVM/Source/llvm' is not latest?
Hey @macmade, I just discovered this project and was really delighted by how easy it was to start using to parse Objective-C files. I understand that you haven't used it in a while, and have moved onto other projects, but I was curious if you had any idea how much work it would be to support reading swift files and getting back an array of tokens?
Currently, the call to -tokens
returns nil
, so it would seem that the call to clang_tokenize
yields zero tokens.
Your installation guide instructs to build the LLVM sources using the makefile.
After running the makefile, what must be done?
I'm getting a bunch of linker errors when compiling ClangKit.
I ran make
in the LLVM directory. Is there anything else I need to do to get these problems solved?
By the way I'm on Xcode 5.0.1 and OS X 10.9.
@" NSString * x;\n"
ok so we know where this is...
"<CKToken: 0x100b1aed0>: Identifier[9:5] NSString
Let's try a completion on the next line...
@" NSStrin\n"
NSArray *comp = [tu completionResultsForLine:10 column:12];
NSLog(@"count: %ld", comp.count);
Result with Foundation.h
linked.. 33718 without... 232.
That seems like ALL the completions available, to me... Not just those that correspond to this particular spot...
How can we get the results to correspond to the actual request / context?
Hi, I'm trying to build ClangKit by checking out the repo and running make
from ClangKit/LLVM
folder
Once compilation is finished (with no visible errors) I'm opening ClangKit.xcodeproj
and straightaway it has libclangRewriteCore.a
missing both from Libraries/Release
and Libraries/Debug
groups (as well as from the compiled files on the disk).
Any ideas what might cause this issues?
Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.