juliainterop / clang.jl Goto Github PK

It seems that Clang.jl is dependent on BinDeps.jl, but the package manager doesn't know it yet. Bit me on a fresh Julia install.

• The '_t' types - http://en.wikipedia.org/wiki/C_data_types#Fixed_width_integer_types mapped to appropriate fixed width Julia types
• size_t mapped to Csize_t
• ptrdiff_t mapped to Cptrdiff_t

They are commonly defined in standard C headers which will be excluded from processing while using wrap_c. Hence makes sense to map them to Julia types as an in-built mapping.

A similar argument can be made for C date/time and FILE types, but they are relatively rarer compared to the integer/size types.

tu_init, tu_parse, others?

• tests
• better dependency resolution (emit types after the entire pass)
• integrate new struct improvements
• support const array definitions

This will make the generator scripts easier to understand/replicate.

The typedef'd type names are good to have, but original arg names would also be very useful (ie:
https://groups.google.com/d/msg/julia-dev/mYCVcmM-dcQ/Dw2PIW5sSDkJ )

The pattern below is relatively common in header files:

typedef struct mysecretstruct SECRET;

void foo(SECRET*);

without any definition of mysecretstruct.

Right now, wrap_c will output

WARNING: Skipping empty struct: "mysecretstruct"

and produce:

function foo(::Ptr{SECRET})
  ccall( (:foo, test), None, (Ptr{SECRET},), )
end

which doesn't quite work because SECRET isn't defined anywhere in Julia. It seems like the right thing to do is produce Ptr{Void} here instead?

The wip_cpp branch is now sufficiently developed to enable this proof-of-concept demo:

https://github.com/ihnorton/VTK.jl

C++ method calls are supported as follows:

• static methods: normal ccall, using mangled name, with signature like:
  @Scall Ptr{Void} New () _ZN11vtkPolyData3NewEv "libvtkFiltering"

  macro: https://github.com/ihnorton/Clang.jl/blob/wip_cpp/src/wrap_cpp.jl#L37-L48

• member methods: called by mangled name using thiscall, passing this* first. Sig:
  @mCall None SetPolys (Ptr{Void},) _ZN11vtkPolyData8SetPolysEP12vtkCellArray "libvtkFiltering"

  macro: https://github.com/ihnorton/Clang.jl/blob/wip_cpp/src/wrap_cpp.jl#L50-L61

• virtual methods: the vtable index of the function is annotated during generation (probably only supports single-inheritance for now). In each function call, the appropriate function pointer is retrieved by de-referencing the this* object to get the vtable, then de-referencing again at the appropriate offset (= index * sizeof(Ptr))

  sig:
  @Vcall 90 None GetCellNeighbors (vtkIdType, Ptr{Void}, Ptr{Void})

  macro:
  https://github.com/ihnorton/Clang.jl/blob/wip_cpp/src/wrap_cpp.jl#L63-L74

There are a lot of things to consider in developing this to the point where it is usable, so this issue and the c++ tag will be used to capture and discuss.

Building it fails because of a missing include (cstdio).

I'm using g++ 4.7.1.
I think this didn't cause an error for others because older C++ headers weren't properly sanitized and included cstdio by themselves.

Build error:

g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I"/local/matthies/src-others/julia/julia.git//usr/include" -g -Wall -Wno-strict-aliasing -fno-omit-frame-pointe
r -fPIC 
wrapclang.cpp: In function ‘void wci_debug_token(CXTranslationUnit, char*)’:
wrapclang.cpp:118:69: error: ‘printf’ was not declared in this scope
wrapclang.cpp: In function ‘void wci_print_tokens(CXTranslationUnit, char*)’:
wrapclang.cpp:127:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
wrapclang.cpp:131:71: error: ‘printf’ was not declared in this scope
make: *** [wrapclang.cpp.o] Error 1

Fixed by following patch

--- wrapclang.cpp.bak   2014-02-04 14:40:54.974952085 +0100
+++ wrapclang.cpp       2014-02-04 14:38:17.951127636 +0100
@@ -1,6 +1,7 @@
 #include <cstring>
 #include <vector>
 #include <set>
+#include <cstdio>

 extern "C" {
 #include "clang-c/Index.h"

The line:

#define SCIP_EVENTTYPE_SOLFOUND       (SCIP_EVENTTYPE_POORSOLFOUND | SCIP_EVENTTYPE_BESTSOLFOUND)

is translated to

# Skipping MacroDefinition: SCIP_EVENTTYPE_SOLFOUND(SCIP_EVENTTYPE_POORSOLFOUND|SCIP_EVENTTYPE_BESTSOLFOUND)

There seems to be a lot of missing auto-conversion methods. I keep running into places where I get no-method errors, but have no idea what function call I'm supposed to wrap it in first. For example, that's a lot of functions to call to find out that the time_t being returned from my function is a Long (on ubuntu-64):

julia> cindex.getTypedefDeclUnderlyingType(cindex.getTypeDeclaration(cindex.getTypedefDeclUnderlyingType(cindex.getTypeDeclaration(cindex.return_type(fdecl)))))
CLType (Long) 

Yes, I could just do the following. But, I wanted to explore the type hierarchy, as a means of understanding the layout of this header file, and as a precursor to attempting my own (highly specialized) C-to-Julia converter.

julia> cindex.getCanonicalType(cindex.return_type(fdecl))

Even the examples seem to encounter this. They seem to have a lot of boiler-plate code that seems to be almost generic enough to be included in the source code. (and perhaps just shown in the examples as reference material)
http://nbviewer.ipython.org/urls/raw.github.com/ihnorton/Clang.jl/master/examples/example_notebook.ipynb

@isaiah,

I am trying to install Clang.jl via Pkg.add() and I am getting the following error:

julia> Pkg.add("Clang")
MESSAGE: Installing Clang v0.0.0
MESSAGE: Running build script for package Clang
g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I"/home/amitm/Work/julia/julia/usr/include" -g -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fPIC
In file included from wrapclang.cpp:37:0:
wrapclang.h: In function ‘void wci_getTypedefDeclUnderlyingType(char_, char_)’:
wrapclang.h:93:52: error: ‘clang_getTypedefDeclUnderlyingType’ was not declared in this scope
wrapclang.h: In function ‘void wci_getEnumDeclIntegerType(char_, char_)’:
wrapclang.h:98:46: error: ‘clang_getEnumDeclIntegerType’ was not declared in this scope
wrapclang.h: In function ‘long long int wci_getEnumConstantDeclValue(char_)’:
wrapclang.h:103:43: error: ‘clang_getEnumConstantDeclValue’ was not declared in this scope
wrapclang.h: In function ‘long long unsigned int wci_getEnumConstantDeclUnsignedValue(char_)’:
wrapclang.h:107:51: error: ‘clang_getEnumConstantDeclUnsignedValue’ was not declared in this scope
wrapclang.h: In function ‘int wci_Cursor_getNumArguments(char_)’:
wrapclang.h:111:41: error: ‘clang_Cursor_getNumArguments’ was not declared in this scope
wrapclang.h: In function ‘void wci_Cursor_getArgument(char_, int, char_)’:
wrapclang.h:115:47: error: ‘clang_Cursor_getArgument’ was not declared in this scope
wrapclang.h: In function ‘int wci_getFunctionTypeCallingConv(char_)’:
wrapclang.h:161:45: error: ‘clang_getFunctionTypeCallingConv’ was not declared in this scope
wrapclang.h: In function ‘int wci_getNumArgTypes(char_)’:
wrapclang.h:170:33: error: ‘clang_getNumArgTypes’ was not declared in this scope
wrapclang.h: In function ‘void wci_getArgType(char_, unsigned int, char_)’:
wrapclang.h:174:37: error: ‘clang_getArgType’ was not declared in this scope
wrapclang.h: In function ‘unsigned int wci_isFunctionTypeVariadic(char_)’:
wrapclang.h:179:41: error: ‘clang_isFunctionTypeVariadic’ was not declared in this scope
wrapclang.h: In function ‘void wci_getElementType(char_, char_)’:
wrapclang.h:192:38: error: ‘clang_getElementType’ was not declared in this scope
wrapclang.h: In function ‘long long int wci_getNumElements(char_)’:
wrapclang.h:197:33: error: ‘clang_getNumElements’ was not declared in this scope
wrapclang.h:198:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘unsigned int wci_isFunctionTypeVariadic(char_)’:
wrapclang.h:180:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘int wci_getNumArgTypes(char_)’:
wrapclang.h:171:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘int wci_getFunctionTypeCallingConv(char_)’:
wrapclang.h:162:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘int wci_Cursor_getNumArguments(char_)’:
wrapclang.h:112:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘long long unsigned int wci_getEnumConstantDeclUnsignedValue(char_)’:
wrapclang.h:108:1: warning: control reaches end of non-void function [-Wreturn-type]
wrapclang.h: In function ‘long long int wci_getEnumConstantDeclValue(char_)’:
wrapclang.h:104:1: warning: control reaches end of non-void function [-Wreturn-type]
make: *_* [wrapclang.cpp.o] Error 1
ERROR: failed process: Process(make, ProcessExited(2)) [2]
in pipeline_error at process.jl:394
in run at process.jl:384
in include_from_node1 at loading.jl:89
in anonymous at no file:241
in cd at file.jl:26
in runbuildscript at pkg.jl:238
in _resolve at pkg.jl:314
in anonymous at no file:155
in cd at file.jl:26
in cd_pkgdir at pkg.jl:34
in add at pkg.jl:135
in add at pkg.jl:167
at /home/amitm/.julia/Clang/deps/build.jl:4

void SCIPmessageVPrintError(const char* formatstr,   va_list ap);

produces

function SCIPmessageVPrintError(formatstr::Ptr{Uint8}, ap::va_list)
  ccall( (:SCIPmessageVPrintError, libscip), None, (Ptr{Uint8}, va_list), formatstr, ap)
end

Should this function just be ignored until Julia can handle va_list?

Goal is to work with packaged versions of various C++ libraries. No recompilation of the library should be required. Nothing should be needed on the Julia side except the headers.

• verify that the table layout is generated correctly by clang in light of inlining (vs gcc)
• function to resolve header includes to real paths
• generate C wrapper for all public inline functions.

Julia's string rules are similar to C, but we need to escape $ for better results:
tok = replace(tok, "\$", "\\\$")

For usability, and to avoid generating an excessive number of functions, it will be necessary to include inheritance information on the Julia side so that inherited methods (that are not overridden) can be called with derived types.

• Julia supports dispatching on pointer type. Is it possible to dispatch Ptr{child} types to Ptr{parent} methods?
• if so then generator will need to create the appropriate (single-inheritance..) hierarchy of dummy types
• then functions will need to be created with specific type for this*
• note: for VTK (and probably others), it is necessary to customize the return type of the constructor (::New for VTK) so that the object pointer has the appropriate type (right now everything defaults to Void). This should just take some modification of ctype_to_julia to return typealias'd name and avoid de-referencing down to intrinsic type.

I've been trying to install Clang in Julia for a while, but without success. When I do Pkg.add("Clang"), I get the following warning:

ERROR: Unknown package Clang; Perhaps you need to Pkg.update() for new metadata?
in anonymous at no file:140
in cd at file.jl:26
in cd_pkgdir at pkg.jl:34
in add at pkg.jl:135
in add at pkg.jl:167

I've tried doing Pkg.update() (It shows: Already up-to-date.) I've also tried removing ~/.julia and then adding the package, it doesn't work. Can someone please help me on this? I'm on Ubuntu 13.04, and my julia version is 0.1.2. I've also installed clang-3.2 and libclang-common-dev on my system.

the printobj() function in you example notebook seems really great for interactive development ("I've parsed my header, now what does it contain?"). could you add it to cindex, with the name dump()?

~/julia (master ✓) julia
_
_ _ ()_ | A fresh approach to technical computing
() | () () | Documentation: http://docs.julialang.org
_ _ | | __ _ | Type "help()" to list help topics
| | | | | | |/ ` | |
| | || | | | (| | | Version 0.2.0-1796.rbac428f9
/ |_'|||__'| | Commit bac428f990 2013-06-03 15:11:03
|__/ |

julia> Pkg.add("Clang")
MESSAGE: Installing Clang v0.0.0
in anonymous at no file:163
Cloning into 'Clang'...
remote: Counting objects: 599, done.
remote: Compressing objects: 100% (298/298), done.
remote: Total 599 (delta 337), reused 547 (delta 289)
Receiving objects: 100% (599/599), 119.41 KiB | 0 bytes/s, done.
Resolving deltas: 100% (337/337), done.
MESSAGE: Running build script for package Clang
in cd at file.jl:25
Makefile:7: *** Environment variable JULIAHOME is not set.. Stop.
ERROR: failed process: Process(make, ProcessExited(2)) [2]
in error at error.jl:22
in pipeline_error at process.jl:400
in run at process.jl:390
in evalfile at loading.jl:145
in anonymous at no file:249
in cd at file.jl:25
in runbuildscript at pkg.jl:246
in _resolve at pkg.jl:394
in anonymous at no file:163
in cd at file.jl:25
in cd_pkgdir at pkg.jl:42
in add at pkg.jl:143
in add at pkg.jl:175

julia> exit()

The function Base.find_library already does what your find_library function does:

diff --git a/deps/ext.jl b/deps/ext.jl
index dc93145..5cb4405 100644
--- a/deps/ext.jl
+++ b/deps/ext.jl
@@ -1,16 +1,4 @@
-let 
-    function find_library(libname,filename)
-        try 
-            push!(DL_LOAD_PATH, joinpath(Pkg.dir(), "Clang", "deps", "usr", "lib"))
-            dl = dlopen(libname)
-        catch
-            try 
-                dl = dlopen(libname)
-                dlclose(dl)
-            catch
-                error("Failed to find required library "*libname*". Try re-running the package script using Pkg.build(\"pkg\")")
-            end
-        end
-    end
-    find_library("libwrapclang","libwrapclang")
+const libwci = find_library(["libwrapclang",],[Pkg.dir("Clang", "deps", "usr", "lib"),]) 
+if libwci == ""
+    error("Failed to find required library libwrapclang. Try re-running the package script using Pkg.build(\"Clang\")")
 end

edit: save the libwci constant

In wrapping libcudart, I noticed that almost every single function returns a cudaError_t. It would be convenient to wrap ccalls with this return type in a function that checks the error code and then does the right thing. Obviously I can do a search-and-replace for the ccall, but I wonder if you've considered giving the user some control over how each function is written out?

In particular, it seems that one might also pass the WrapContext to each wrap function, and then have an extra field, outerwrapper, that can store a function

pre_post(funcname, ret_type, arg_types)

A default value would be

pre_post(funcname, ret_type, arg_types) = ("", "")

but in my case it might be

function pre_post(funcname, ret_type, arg_types)
    if ret_type == "cudaError_t"
        return "checkerror(", ")"
    end
    ("", "")
end

and the pre/post strings get put before/after the ccall.

I could imagine this getting more complicated. I might want to have a "don't wrap" list for StructDecls. Then you might want to pass the cursor to the outerwrapper function so that it can dispatch on it.

Obviously I could simply overwrite wrap_c.wrap((buf::IO, funcdecl::FunctionDecl, libname::ASCIIString), but I'd of course have to copy most of it, which is why I thought of adding a hook. But this may be a terrible design or have terrible names, hence I'm first asking you in an issue rather than submit a PR.

wrap_c will need a better mechanism to decide which header files to actually include.

The first question is: should we make typealias (typedef) available on the Julia side? It is possible to just resolve all the typedefs to the real underlying type at generation time, but I think using typedefs is much preferred: make it look like the original API so that those docs can be referenced.

Considering above, then we need some way to check inclusion criteria for headers during generation. Right now, tu_parse will just start at the first definition in the first included file, and we end up wrapping extraneous stuff. However, we can't restrict to the top include file, because things like libgit2 use a dummy header that just includes a bunch of stuff in subfolder.

The only sane way to handle this may be to allow the user to pass in a func::Bool which will check inclusion for each header based on some project-specific heuristic.

dlopen should happen only once per library in the generated code. Need to generate a list of required libraries during generation, initialize those in the common file, and pass the appropriate (const) pointer into each macro.

aviks ~/dev/julia $ ./julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" to list help topics
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.3.0-prerelease+855 (2014-01-07 15:28 UTC)
 _/ |\__'_|_|_|\__'_|  |  master/6e4ed01* (fork: 4 commits, 0 days)
|__/                   |  x86_64-apple-darwin11.4.2

julia> using Clang

julia> context=wrap_c.init(output_file="$(joinpath(Pkg.dir(), "YAML","src","libyaml.jl"))", header_library="libyaml")
WrapContext(Ptr{Void} @0x000000010ab70f00,"/Users/aviks/.julia/YAML/src/libyaml.jl","/Users/aviks/.julia/YAML/src/libyaml.jl",[],[],(anonymous function),(anonymous function),(anonymous function),None,Set{ASCIIString}(),Dict{ASCIIString,IO}(),InternalOptions(false),0)

julia> headers=[joinpath(Pkg.dir(),"YAML", "deps", "yaml-0.1.4", "include", "yaml.h")]
1-element Array{ASCIIString,1}:
 "/Users/aviks/.julia/YAML/deps/yaml-0.1.4/include/yaml.h"

julia> wrap_c.wrap_c_headers(context, headers)

WRAPPING HEADER: /Users/aviks/.julia/YAML/deps/yaml-0.1.4/include/yaml.h
Wrap: CLCursor (TypeRef) struct __darwin_sigaltstack
Wrap: CLCursor (TypeRef) struct __darwin_ucontext
ERROR: no method wrap(IOStream, FunctionDecl, Function)
 in wrap_header at /Users/aviks/.julia/Clang/src/wrap_c.jl:413
 in wrap_c_headers at /Users/aviks/.julia/Clang/src/wrap_c.jl:476

using master of Clang

On MACOS:
require("Clang") gives this:
ERROR: wrong number of arguments
in include at boot.jl:238
in include_from_node1 at loading.jl:96
in include at boot.jl:238
in include_from_node1 at loading.jl:96
in reload_path at loading.jl:121
in require at loading.jl:50
at /Users/dominicsuciu/.julia/Clang/src/../deps/ext.jl:1
at /Users/dominicsuciu/.julia/Clang/src/Clang.jl:6

This fails whether its run during Pkg add or at command line:
g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I"/Applications/JuliaStudio.app//usr/include" -g -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fPIC

wrapclang.cpp:6:27: error: clang-c/Index.h: No such file or directory
wrapclang.cpp:17: error: variable or field ‘wci_save_CXSourceLocation’ declared void
wrapclang.cpp:17: error: ‘CXSourceLocation’ was not declared in this scope
wrapcla . . . . etc

ERROR: no method sort(KeyIterator{Dict{ASCIIString,Int64}},)
in sort_common_includes at /home/tkeitt/.julia/Clang/src/wrap_c.jl:373
in wrap_c_headers at /home/tkeitt/.julia/Clang/src/wrap_c.jl:440
in evalfile at loading.jl:144

I was wondering why I was not getting a "common" file. This happens right before the code block that writes the common file.

@timholy's multi-library script demonstrates the need to support mapping from multiple headers to different shared libraries. This is similar and complementary to the need to specify multiple output files so that header definitions can be automatically grouped into modules.

Can wrap_c.init() take in an additional callback function which can be used to determine if a function should also be exported and the corresponding export statement generated?

given this signature:

xmlStrVPrintf (xmlChar *buf, int len, const xmlChar *msg, va_list ap);

clang resolves the last argument to "__va_list_tag", which is a typedef'd struct. Julia appears to support this in ccall with "..."

We need to detect vararg and add the varargs specifier to the generated ccall.

This looks really amazing. I'm testing it to see if I can wrap libavcodec. I am following the model of wrap_libXML2.jl. I found that I had to delete the {} argument of wrap_c_headers, and I inserted an extra println() to see which header files are parsed.

Here's where I'm stuck now:

julia> include("wrap_libav.jl")
["/usr/include/libavcodec/avcodec.h", "/usr/include/libavcodec/avfft.h"  …  "/usr/include/libavcodec/version.h", "/usr/include/libavcodec/xvmc.h"]
clang args: {"-I", "/home/tim/src/julia/deps/llvm-3.2/build/Release/lib/clang/3.2/include", "-I"  …  "-I", "/usr/include/libavcodec/", check_use_header}
ERROR: no method tu_parse(Ptr{None},ASCIIString,Array{Any,1})
 in wrap_c_headers at /home/tim/.julia/Clang/src/wrap_c.jl:130
 in include_from_node1 at loading.jl:76
at /home/tim/src/julia-modules/libav/src/wrap_libav.jl:28

julia> cindex.tu_parse
# methods for generic function tu_parse
tu_parse(Any,ASCIIString) at /home/tim/.julia/Clang/src/cindex.jl:123
tu_parse(Any,ASCIIString,Array{ASCIIString,1}) at /home/tim/.julia/Clang/src/cindex.jl:124
tu_parse(Any,ASCIIString,Array{ASCIIString,1},Any,Ptr{None},Any,Any) at /home/tim/.julia/Clang/src/cindex.jl:126

The vtable layout appears to be the same between linux and windows (mingw g++, reasonably close to linux version)

Need to find someone to test this on a mac, or give me remote access to a box for a few hours.

@c Ptr{CURL} curl_easy_init () :libcurl

used as

    curl = curl_easy_init()

results in the following error.

ERROR: type: get: in ccall: function argument not a pointer or valid, constant expression, expected DataType, got (DataType,)

Can wrap_c.init() take in an additional callback function which can be used to support varargs? i.e, can the callback function specify a list of possible vararg sets?
For example:

    CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...); 

results in the wrong wrapper for curl_easy_setopt, i.e., a definition that takes in only the first 2 parameters. But this is mostly harmless.

But, it would be great if I could specify additional possible usage definitions like by providing a list of possible argument types (for the varargs) as sublists like this - [[Int32, Ptr{:Uint8}], [Int32], [ Ptr{:Uint8}]].

For example the possible actual usage scenarios for curl_easy_setopt are

CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, long val);
CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, char * val);
CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, void * p);

So, in the above case I could just specify the possibilities for the 3rd parameter.

this is probably going to be limited and require some hoop-jumping, but should utilize as much as the libclang api exposes.

julia> Pkg.build("Clang")
INFO: Building Clang
g++ wrapclang.cpp -fPIC -c -o wrapclang.cpp.o -I"/Users/stefan/projects/julia//usr/include" -g -Wall -Wno-strict-aliasing -fno-omit-frame-pointer -fPIC
wrapclang.cpp:6:10: fatal error: 'clang-c/Index.h' file not found
#include "clang-c/Index.h"
         ^
1 error generated.
make: *** [wrapclang.cpp.o] Error 1
================================[ ERROR: Clang ]================================

failed process: Process(`make`, ProcessExited(2)) [2]
while loading /Users/stefan/.julia/Clang/deps/build.jl, in expression starting on line 6

================================================================================

================================[ BUILD ERRORS ]================================

WARNING: Clang had build errors.

 - packages with build errors remain installed in /Users/stefan/.julia
 - build a package and all its dependencies with `Pkg.build(pkg)`
 - build a single package by running its `deps/build.jl` script

================================================================================

Not sure where to start debugging this.

see CurKind.integerliteral, and CurKind.EnumConstantDecl

function_args() provides incomplete type information (see below) and doesn't have parameter names

a more useful definition would seem to be:
function_args(f) = filter!((f)->isa(f, ParmDecl),[children(fdecl)...])

julia> fdecl2
CLCursor (FunctionDecl) g_object_set_valist(GObject *, const gchar *, __va_list_tag *)

julia> function_args(fdecl2)[3]
CLType (Pointer) 

julia> getTypeDeclaration(getCursorType([children(fdecl2)...][3]))
CLCursor (TypedefDecl) va_list

I am using Clang.jl to generate wrappers for libCURL using the following code:

using Clang.cindex
using Clang.wrap_c

JULIAHOME=EnvHash()["JULIAHOME"]

clang_includes = map(x->joinpath(JULIAHOME, x), [
  "deps/llvm-3.2/build/Release/lib/clang/3.2/include",
  "deps/llvm-3.2/include",
  "deps/llvm-3.2/include",
  "deps/llvm-3.2/build/include/",
  "deps/llvm-3.2/include/"
  ])
clang_extraargs = ["-D", "__STDC_LIMIT_MACROS", "-D", "__STDC_CONSTANT_MACROS"]

wrap_hdrs = map( x-> joinpath("/usr/include/curl", x), [ "curl.h", "easy.h", "multi.h" ])

wc = wrap_c.init(".", "lC_common_h.jl", clang_includes, clang_extraargs, (th, h) -> contains(wrap_hdrs, h) , h -> ":libcurl", h -> "./lC_" * replace(last(split(h, "/")), ".", "_")  * ".jl" )
wrap_c.wrap_c_headers(wc, ["/usr/include/curl/curl.h"])

The issue: :

@ctypedefs for enum types did not get generated for non-top level headers. For example,
"@ctypedef CURLcode Int32" was generated for CURLcode(defined in top-level header curl.h) but the equivalent for CURLMcode (defined in included multi.h) was not generated

Generate Julia types to match C struct declarations

• fixed-size arrays
• unions

Clang seems to pick up and wrap tons of low-level functions like printf, fread, sin, cos, etc. Can I filter the output? All exported symbols in the particular library I'm working with begin with a particular pattern, so it would be easy to check if I could provide a filter function.

The first line of http://clangjl.readthedocs.org/en/latest/wrap_c.html, now fails like this:

julia> using Clang

julia> context = wrap_c.init()
ERROR: Missing header_library argument: pass lib name, or (hdr)->lib::ASCIIString function
 in init at /usr/local/julia/julia-packages/Clang/src/wrap_c.jl:72

(This isn't getting in my way, I just thought you should know.)

For me trying to run the code in example/example_notebook.ipynb results in a complaint about the clang_includes named argument to parse_header. Should that name be includes?

Also, for recent versions of julia (I haven't gone back to 0.2 to check if this is true there too) the clang-c directory gets installed in abspath(JULIA_HOME,"../include") if I set BUILD_LLVM_CLANG=1, which seems a more reliable path than one with the version number embedded.

This is a little puzzling: in the generated code,

type cudaDeviceProp
    name::Array_256_Uint8
    totalGlobalMem::Cint
    ... (54 fields in total)

But cuda_runtime_api.h declares this as

/**
 * \brief Returns information about the compute-device
 *
 * Returns in \p *prop the properties of device \p dev. The ::cudaDeviceProp
 * structure is defined as:
 * \code
    struct cudaDeviceProp {
        char name[256];
        size_t totalGlobalMem;
        ...

size_t should resolve to a Csize_t, not a Cint. Its cu_type is Type: CLType (Typedef).

The inheritance machinery needs to detect when a parent class is templated, and figure out what to do.

Example: vtkDoubleArray, inherits from vtkDataArrayTemplate<> ... Maybe just inherit from the superclass of vtkDataArrayTemplate?

My copy of clang has trouble finding it's builtin headers if it isn't started from the usr/lib directory (actually, it expected usr/bin aka the location of the clang executable, but lib will work just as well). Not certain if there's anything you can do about this, but chdir works for me.

julia> top=cindex.parse_header("",args=["-v"])
clang version 3.3 (tags/RELEASE_33/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
ignoring nonexistent directory "../lib/clang/3.3/include"

#define foo(x) x

produces

const foo = (x)

and obviously x isn't defined.

Sorry for the barrage of issues here, trying to use Clang to wrap a pretty large code base.

This is really just an FYI. I was able to get everything working (at least, at the level of being able to include the files without error---I haven't actually tested code yet) by adding following manually-generated file:

# These are all used only inside Ptrs
typealias cudaArray Void
typealias cudaMipmappedArray Void
typealias CUstream_st Void
typealias CUevent_st Void
typealias cudaGraphicsResource Void

include("libcudart_h.jl")
typealias cudaError_t cudaError
# Important! Delete or comment-out functions in libcudart.jl using
# cudaResourceDesc

Explanations:

• During wrapping, I see clear messages like WARNING: Skipping empty struct: "cudaArray". EDIT: this makes it easy to figure out which ones I needed to add a typealias ... Void to, once I knew the pattern.
• The issue with cudaResourceDesc is that it has union members that are structs. It was converted as:

type cudaResourceDesc
    resType::cudaResourceType
    res::
end

I had to comment it out in libcudart_h.jl, and then comment out any functions using it in libcudart.jl.

Those are pretty minimal manual interventions. Quite an amazing tool, Isaiah!

Right now vtable function pointers are retrieved using:

derefptr(thisptr, vtidx) = pointer(Void, unsafe_ref(pointer(Uint64, unsafe_ref(pointer(Uint64,thisptr)))+vtidx*8) )

The IR looks like this:

derefptr(thisptr, vtidx) = pointer(Void, unsafe_ref(pointer(Uint64, unsafe_ref(pointer(Uint64,thisptr)))+vtidx*8) )

julia> disassemble(derefptr, (Ptr{Void},Int64))

define i8* @julia_derefptr817(i8*, i64) {
top:
  %2 = bitcast i8* %0 to i64*, !dbg !7020, !julia_type !7027
  %3 = load i64* %2, !dbg !7020, !julia_type !7028
  %4 = inttoptr i64 %3 to i64*, !dbg !7020, !julia_type !7027
  %5 = ptrtoint i64* %4 to i64, !dbg !7020, !julia_type !7028
  %6 = shl i64 %1, 3, !dbg !7020
  %7 = add i64 %5, %6, !dbg !7020, !julia_type !7028
  %8 = inttoptr i64 %7 to i64*, !dbg !7020, !julia_type !7027
  %9 = load i64* %8, !dbg !7020, !julia_type !7028
  %10 = inttoptr i64 %9 to i8*, !dbg !7020, !julia_type !7029
  ret i8* %10, !dbg !7020
}

Not sure what this compiles down to, but looking at disassembly of some static code, I think it should optimally take three instructions (mov, mov, call).