trolldbois / ctypeslib Goto Github PK
View Code? Open in Web Editor NEWGenerate python ctypes classes from C headers. Requires LLVM clang
Home Page: http://trolldbois.blogspot.com/search?q=ctypeslib
License: MIT License
Generate python ctypes classes from C headers. Requires LLVM clang
Home Page: http://trolldbois.blogspot.com/search?q=ctypeslib
License: MIT License
Hello, I was trying to use the ctypeslib, installed from master branch and python-clang==3.5
When running clang2py NBioAPI_IndexSearch.h
result in error
Do you have any idea what might be going wrong?
If help on something, the header that I was testing this is here: https://github.com/luzfcb/nbiosearch/blob/master/nbiosearch/eNBSP/include/NBioAPI_IndexSearch.h
sutransdev@sutransdev:~/nbiosearch/nbiosearch/eNBSP/include$ clang2py NBioAPI_IndexSearch.h
WARNING:clangparser:Source code has some error. Please fix.
WARNING:clangparser:unknown type name 'wchar_t'
Traceback (most recent call last):
File "/usr/local/bin/clang2py", line 6, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/clang2py.py", line 235, in main
flags=clang_opts)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/codegenerator.py", line 777, in generate_code
parser.parse(srcfile)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 114, in parse
self.startElement( node )
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 125, in startElement
stop_recurse = self.parse_cursor(node)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 232, in parse_cursor
return self.cursorkind_handler.parse_cursor(cursor)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
return mth(cursor)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/util.py", line 114, in fn
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 498, in STRUCT_DECL
return self._record_decl(cursor, typedesc.Structure)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/cursorhandler.py", line 553, in _record_decl
fields = list(cursor.type.get_fields())
AttributeError: 'Type' object has no attribute 'get_fields'
Could be a mix of a few different bugs in here.
[ 57s] ======================================================================
[ 57s] ERROR: test_char (test_types_values.ConstantsTest)
[ 57s] ----------------------------------------------------------------------
[ 57s] Traceback (most recent call last):
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 123, in test_char
[ 57s] """)
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 168, in convert
[ 57s] self.gen(hfile, flags, debug)
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 153, in gen
[ 57s] exec(output, namespace)
[ 57s] File "<string>", line 12, in <module>
[ 57s] NameError: name 'x' is not defined
[ 57s]
[ 57s] ======================================================================
[ 57s] FAIL: test_includes (test_fast_clang.CompareSizes)
[ 57s] Test sizes of pod with std include.
[ 57s] ----------------------------------------------------------------------
[ 57s] Traceback (most recent call last):
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_fast_clang.py", line 54, in test_includes
[ 57s] self.assertSizes(name)
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 189, in assertSizes
[ 57s] name)
[ 57s] AssertionError: False is not true : int8_t was not found in source
[ 57s]
[ 57s] ======================================================================
[ 57s] FAIL: test_array (test_types_values.ConstantsTest)
[ 57s] ----------------------------------------------------------------------
[ 57s] Traceback (most recent call last):
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 163, in test_array
[ 57s] self.assertEqual(self.namespace.c1, [])
[ 57s] AssertionError: <class 'c_ubyte_Array_1'> != []
[ 57s]
[ 57s] ======================================================================
[ 57s] FAIL: test_char_p (test_types_values.ConstantsTest)
[ 57s] ----------------------------------------------------------------------
[ 57s] Traceback (most recent call last):
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 136, in test_char_p
[ 57s] self.assertEqual(self.namespace.x, [])
[ 57s] AssertionError: <class 'c_ubyte_Array_10'> != []
[ 57s]
[ 57s] ======================================================================
[ 57s] FAIL: test_extern_function_pointer (test_types_values.ConstantsTest)
[ 57s] ----------------------------------------------------------------------
[ 57s] Traceback (most recent call last):
[ 57s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 271, in test_extern_function_pointer
[ 57s] 'LP_c_char')
[ 57s] AssertionError: 'LP_c_ubyte' != 'LP_c_char'
[ 57s] - LP_c_ubyte
[ 57s] + LP_c_char
Hi. I am trying to create bindings for libjxr. I get some output that looks somehow usable, but the structures have no useful attributes, it all looks like:
PKFactory = struct_tagPKFactory
class struct_tagPKCodecFactory(ctypes.Structure):
_pack_ = True # source:False
_fields_ = [
('PADDING_0', ctypes.c_ubyte),
]
PKCodecFactory = struct_tagPKCodecFactory
class struct_tagPKImageDecode(ctypes.Structure):
_pack_ = True # source:False
_fields_ = [
('PADDING_0', ctypes.c_ubyte),
]
PKImageDecode = struct_tagPKImageDecode
class struct_tagPKImageEncode(ctypes.Structure):
_pack_ = True # source:False
_fields_ = [
('PADDING_0', ctypes.c_ubyte),
]
PKImageEncode = struct_tagPKImageEncode
Any hints?...
Parsing radare2 header and have this error:
2017-12-28 12:23:23,355 'r_userconf.h' file not found with <angled> include; use "quotes" instead
2017-12-28 12:23:23,355 Source code has some error. Please fix.
2017-12-28 12:23:23,355 'r_userconf.h' file not found with <angled> include; use "quotes" instead
When running the following through clang2py
#define PACK __attribute__((packed,aligned(2)))
typedef struct {
long a;
short b;
char c;
double d;
} PACK test;
I get this as ouput
class struct_c__SA_test(ctypes.Structure):
_pack_ = True # source:True
_fields_ = [
('a', ctypes.c_int32),
('b', ctypes.c_int16),
('c', ctypes.c_ubyte),
('d', ctypes.c_double),
('PADDING_0', ctypes.c_ubyte),
]
test = struct_c__SA_test
In the ctypes docs is says that pack should be a number (of bytes). When run through the original h2xml/xml2py the pack attribute is set to 2, as expected. Shouldn't this be the behavior of ctypeslib2 as well?
https://github.com/trolldbois/ctypeslib/blob/master/ctypeslib/codegen/codegenerator.py#L736
archtypes should be generated, or generable
ctypeslib/ctypeslib/codegen/codegenerator.py
Line 529 in 4220209
I'm sorry, but I don't understand why to set pack to True always.
This will end up with wrong offset & sizeof.
The header files I am trying to process with clang2py.py seem to fail because of a multi-line #define macro definition (where each line, excluding the last one, ends with a backslash \ character). If I comment out this macro and instead copy and paste the definition into each location the macro was used, the code is processed normally. Are multi-line #defines not supported? Would it be feasible to add this support?
setuptools automatically includes test/test*.py
, but omits test/__init__.py
, despite unittest needing a __init__.py
to work on Python 2.
Very annoying, but anyways the __init__.py
has some import bits needed, so needs to be included.
run:
clang2py FOO.h --preload /home/myuser/libMYLIB.so -k dfest -d -o pymylib.py
# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes
CDLL('/home/myuser/libMYLIB.so', RTLD_GLOBAL)
....
I suppose it should be
# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes
ctypes.CDLL('/home/myuser/libMYLIB.so', ctypes.RTLD_GLOBAL)
....
Gives
LibclangError: [WinError 126] The specified module could not be found. To provide a path to libclang use Config.set_library_path() or Config.set_library_file().
both in console and notebook mode.
Can llvm-pdbutil be used to do a PDB-> yaml -> python ctypes ?
https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-pdbutil/YAMLOutputStyle.cpp
int foo;
test.h
:
#define HI(x) x
HI(int) y;
Traceback (most recent call last):
File "/bin/clang2py", line 10, in <module>
sys.exit(main())
File "/lib/python3.6/site-packages/ctypeslib/clang2py.py", line 306, in main
flags=clang_opts)
File "/lib/python3.6/site-packages/ctypeslib/codegen/codegenerator.py", line 888, in generate_code
parser.parse(srcfile)
File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 120, in parse
self.startElement(node)
File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 159, in startElement
stop_recurse = self.parse_cursor(node)
File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
return self.cursorkind_handler.parse_cursor(cursor)
File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
return mth(cursor)
File "/lib/python3.6/site-packages/ctypeslib/codegen/util.py", line 119, in fn
return func(*args, **kwargs)
File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 1041, in MACRO_DEFINITION
tokens = self._literal_handling(cursor)
File "/lib/python3.6/site-packages/ctypeslib/codegen/util.py", line 119, in fn
return func(*args, **kwargs)
File "/lib/python3.6/site-packages/ctypeslib/codegen/cursorhandler.py", line 528, in _literal_handling
value = self.get_registered(value).body
File "/lib/python3.6/site-packages/ctypeslib/codegen/handler.py", line 47, in get_registered
return self.parser.get_registered(name)
File "/lib/python3.6/site-packages/ctypeslib/codegen/clangparser.py", line 188, in get_registered
return self.all[name]
KeyError: 'x'
This generates an error
def test_struct_with_pointer(self):
ns = self.convert('''
struct x {
int y;
};
typedef struct x *x_n_t;
typedef struct p {
x_n_t g[1];
} *p_t;
''')
File "/Library/Python/2.7/site-packages/ctypeslib2-2.0-py2.7.egg/ctypeslib/codegen/clangparser.py", line 224, in get_unique_name
_id = cursor.get_usr()
AttributeError: 'Type' object has no attribute 'get_usr'
Not sure if I’m using a different version of the bindings than you, or what… Do the tests complete?
If a record has an incompletearray, all fields have offset == -2. No padding will be done.
But the fields are ordered and code will be produces with typed info in the right offset.
so in most cases, it will work.
if there is a structure with incompletearray and padding or alignement issue, it will produce wrong results
Bug/corner case is probably in llvm python clang bindings/library libclang
uint8_t buf[2048];
produces
...
File ".../ctypeslib/ctypeslib/codegen/cursorhandler.py", line 400, in _get_var_decl_init_value_single
_v = _v[0]
IndexError: list index out of range
clang2py doesn't generate code for incomplete C structures.
Considering next header file:
#ifndef T_H
#define T_H
struct Foo;
void do_something(struct Foo* foo);
#endif
clang2py issues a few warnings and doesn't generate definitions for struct Foo
and do_something
clang2py -o t.py t.h
ERROR:cursorhandler:Structure struct_Foo is Incomplete t.h:5 align:-2 size:-2
ERROR:cursorhandler:Structure struct_Foo is Incomplete t.h:5 align:-2 size:-2
Generated t.py is empty:
# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes
__all__ = \
[]
Having declared, but not defined structures is most common way in C to hide library internals and it's widely used.
When adding "m" to the -k option, I found that clang2py outputs additional, system-level macros that are missing dependencies. These macros are actually not necessary and do not exist when I was using h2xml/xml2py.
Example:
WCHAR_MIN = __WCHAR_MIN
__ASSERT_FUNCTION = PRETTY_FUNCTION
But the right-hand side symbols are not defined.
If I remove "m" from "-k", these symbols do not exit, but neither do other useful macros.
Is there a way to exclude those useless macros?
Thanks.
In short, ctypeslib2 implements a module named ctypeslib
, which clobbers the legacy ctypeslib
module without providing backwards compatibility. This breaks existing workflows and prevents their straightforward transition from gccxml to clang.
Example:
# h2xml --help
Usage: h2xml includefile ... [options]
Options:
-h, --help show this help message and exit
-q, --quiet
-D NAME[=VALUE] macros to define
-U NAME macros to undefine
-I DIRECTORY additional include directories
-o XMLFILE XML output filename
-c, --cpp-symbols try to find #define symbols - this may give compiler
errors, so it's off by default.
-k don't delete the temporary files created (useful for
finding problems)
#
# pip install ctypeslib2
Downloading/unpacking ctypeslib2
Downloading ctypeslib2-2.1.4.tar.gz (46kB): 46kB downloaded
Running setup.py (path:/tmp/pip-build-dmhvZK/ctypeslib2/setup.py) egg_info for package ctypeslib2
Requirement already satisfied (use --upgrade to upgrade): clang>=3.7 in /usr/local/lib/python2.7/dist-packages (from ctypeslib2)
Installing collected packages: ctypeslib2
Running setup.py install for ctypeslib2
changing mode of build/scripts-2.7/clang2py from 644 to 755
changing mode of /usr/local/bin/clang2py to 755
Successfully installed ctypeslib2
Cleaning up...
#
# h2xml --help
Traceback (most recent call last):
File "/usr/bin/h2xml", line 3, in <module>
from ctypeslib.h2xml import main
ImportError: No module named h2xml
I would suggest that the module provided by ctypeslib2 should be called ctypeslib2
if it is not a functional replacement for the existing ctypeslib
module.
int zig;
inline void foo() {
int zig;
}
Hi, I installed ctypeslib2 via pip: pip install ctypeslib2. It installed clang 6.0 and ctypeslib2, on my windows 10 64 bit, python 3.6 and anaconda.
I have clang 3.9.1 and clangdev installed on my anaconda.
I tried clang2py with an .h file, but receive the following:
function not found clang_CXXRecord_isAbstract. Please check compatibility of libclang.so version.
Now, I believe ctypeslib2 doesn't work on windows.
Any help?
To support commonly installed Clang 8.0, and Clang 9.0.
Sadly this is blocked by the ethanhs/clang#4
need to patch clang so that char[] type is exposed.
your mom gAY
Create a better documentation, examples and list usage for all arguments on the command line
[ 72s] ======================================================================
[ 72s] FAIL: test_extern_function_pointer_multiarg (test_types_values.ConstantsTest)
[ 72s] ----------------------------------------------------------------------
[ 72s] Traceback (most recent call last):
[ 72s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 283, in test_extern_function_pointer_multiarg
[ 72s] 'c_int')
[ 72s] AssertionError: 'c_long' != 'c_int'
[ 72s] - c_long
[ 72s] + c_int
There is a bug which causes unnamed structs and unnamed enums not to be parsed. If you show interest in correcting this issue I will write up a test case which demonstrates the problem.
These tests currently cause (two different) exceptions in clang parser:
def test_extern_function_pointer(self):
ns = self.convert('''
extern int (*func_ptr)(const char *arg);
''')
def test_extern_function_pointer_multiarg(self):
ns = self.convert('''
extern int (*func_ptr)(const char *arg, int c);
''')
typedef struct
{
int x;
} foobar;
extern foobar something;
This errors in Clang_Parser.VAR_DECL because _ctype.KIND == TypeKind.RECORD
is not handled.
const char16_t const_utf16_char = u'あ';
produces
const_utf16_char = あ
Also would be nice to have option to ignore const scalar "variables", since they are almost always inlined.
I am trying to package this for openSUSE Python 3.7, and almost have it passing all tests, only a few remaining issues, which could be my fault, or stuff fixed since 2.2.2 release, or needed enhancements.
[ 23s] test_uint_minus_one (test_types_values.ConstantsTest) ... expected failure
[ 23s] test_unicode (test_types_values.ConstantsTest)
[ 23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[ 23s] Source code has some error. Please fix.
[ 23s] 'stddef.h' file not found
[ 23s] ERROR
[ 23s] test_unicode_cpp11 (test_types_values.ConstantsTest)
[ 23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[ 23s] Source code has some error. Please fix.
[ 23s] 'stddef.h' file not found
[ 23s] ERROR
[ 23s] test_unicode_wchar (test_types_values.ConstantsTest)
[ 23s] unicode conversion test from unittest in clang ... 'stddef.h' file not found
[ 23s] Source code has some error. Please fix.
[ 23s] 'stddef.h' file not found
[ 23s] expected failure
[ 23s] test_var (test_types_values.ConstantsTest)
[ 23s] Basic POD test variable declaration' ... ok
[ 23s] test_var_decl_and_scope (test_types_values.ConstantsTest) ... ok
[ 23s] test_wchar (test_types_values.ConstantsTest) ... ok
[ 23s]
[ 23s] ======================================================================
[ 23s] ERROR: test_unicode (test_types_values.ConstantsTest)
[ 23s] unicode conversion test from unittest in clang
[ 23s] ----------------------------------------------------------------------
[ 23s] Traceback (most recent call last):
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 83, in test_unicode
[ 23s] self.gen('test/data/test-strings.cpp', ['-x', 'c++'])
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 141, in gen
[ 23s] gen = self._gen(ofi, fname, flags)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 129, in _gen
[ 23s] self.parser.parse(fname)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 120, in parse
[ 23s] self.startElement(node)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 159, in startElement
[ 23s] stop_recurse = self.parse_cursor(node)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
[ 23s] return self.cursorkind_handler.parse_cursor(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[ 23s] return mth(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[ 23s] return func(*args, **kwargs)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 249, in VAR_DECL
[ 23s] init_value = self._VAR_DECL_value(cursor, _type)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 299, in _VAR_DECL_value
[ 23s] list(cursor.get_children()))
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[ 23s] _tmp = self._get_var_decl_init_value_single(_ctype, child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 385, in _get_var_decl_init_value_single
[ 23s] child.get_children())
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[ 23s] _tmp = self._get_var_decl_init_value_single(_ctype, child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 394, in _get_var_decl_init_value_single
[ 23s] _v = self.parse_cursor(child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[ 23s] return mth(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[ 23s] return func(*args, **kwargs)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in _literal_handling
[ 23s] str([str(t.spelling) for t in tokens]))
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in <listcomp>
[ 23s] str([str(t.spelling) for t in tokens]))
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 3294, in spelling
[ 23s] return conf.lib.clang_getTokenSpelling(self._tu, self)
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 230, in from_result
[ 23s] return conf.lib.clang_getCString(res)
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 105, in to_python_string
[ 23s] return x.value
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 90, in value
[ 23s] return super(c_char_p, self).value.decode("utf8")
[ 23s] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 1: invalid start byte
[ 23s]
[ 23s] ======================================================================
[ 23s] ERROR: test_unicode_cpp11 (test_types_values.ConstantsTest)
[ 23s] unicode conversion test from unittest in clang
[ 23s] ----------------------------------------------------------------------
[ 23s] Traceback (most recent call last):
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_types_values.py", line 103, in test_unicode_cpp11
[ 23s] self.gen('test/data/test-strings.cpp', ['-x', 'c++', '--std=c++11'])
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 141, in gen
[ 23s] gen = self._gen(ofi, fname, flags)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/util.py", line 129, in _gen
[ 23s] self.parser.parse(fname)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 120, in parse
[ 23s] self.startElement(node)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 159, in startElement
[ 23s] stop_recurse = self.parse_cursor(node)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/clangparser.py", line 273, in parse_cursor
[ 23s] return self.cursorkind_handler.parse_cursor(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[ 23s] return mth(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[ 23s] return func(*args, **kwargs)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 249, in VAR_DECL
[ 23s] init_value = self._VAR_DECL_value(cursor, _type)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 299, in _VAR_DECL_value
[ 23s] list(cursor.get_children()))
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[ 23s] _tmp = self._get_var_decl_init_value_single(_ctype, child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 385, in _get_var_decl_init_value_single
[ 23s] child.get_children())
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 352, in _get_var_decl_init_value
[ 23s] _tmp = self._get_var_decl_init_value_single(_ctype, child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 394, in _get_var_decl_init_value_single
[ 23s] _v = self.parse_cursor(child)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 37, in parse_cursor
[ 23s] return mth(cursor)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/util.py", line 119, in fn
[ 23s] return func(*args, **kwargs)
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in _literal_handling
[ 23s] str([str(t.spelling) for t in tokens]))
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/ctypeslib/codegen/cursorhandler.py", line 463, in <listcomp>
[ 23s] str([str(t.spelling) for t in tokens]))
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 3294, in spelling
[ 23s] return conf.lib.clang_getTokenSpelling(self._tu, self)
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 230, in from_result
[ 23s] return conf.lib.clang_getCString(res)
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 105, in to_python_string
[ 23s] return x.value
[ 23s] File "/usr/lib/python3.7/site-packages/clang/cindex.py", line 90, in value
[ 23s] return super(c_char_p, self).value.decode("utf8")
[ 23s] UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 1: invalid start byte
[ 23s]
[ 23s] ======================================================================
[ 23s] FAIL: test_variable (test_clang2py.ArgumentTypeKind)
[ 23s] run clang2py -k d test/data/test-strings.cpp
[ 23s] ----------------------------------------------------------------------
[ 23s] Traceback (most recent call last):
[ 23s] File "/home/abuild/rpmbuild/BUILD/ctypeslib-2.2.2/test/test_clang2py.py", line 109, in test_variable
[ 23s] self.assertEqual(p.returncode, 0)
[ 23s] AssertionError: 1 != 0
[ 23s]
[ 23s] ----------------------------------------------------------------------
[ 23s] Ran 79 tests in 8.790s
[ 23s]
[ 23s] FAILED (failures=1, errors=2, skipped=4, expected failures=5)
[ 23s] Test failed: <unittest.runner.TextTestResult run=79 errors=2 failures=1>
[ 23s] error: Test failed: <unittest.runner.TextTestResult run=79 errors=2 failures=1>
records should be aware of the PACKED_ATTR.
typedef struct structA {
int x
}structA_t;
struct structB {
structA_t x[8];
};
Problem:
$ pip install ctypeslib2
$ python3
>>> import ctypeslib
No module named 'clang'
Solution:
$ pip install clang
$ python3
>>> import ctypeslib
>>> # ok
Suggested fix:
Add the line from requirements.txt
to setup.py
install_requires
. I did some preliminary investigation to use requirements.txt
from setup.py
and found this.
Consider the following example:
struct example_detail {
int first;
int last;
};
struct example {
int argsz;
int flags;
int count;
struct example_detail details[];
};
From these structs, clang2py
generates:
import ctypes
class struct_example_detail(ctypes.Structure):
_pack_ = True # source:False
_fields_ = [
('first', ctypes.c_int32),
('last', ctypes.c_int32),
]
class struct_example(ctypes.Structure):
_pack_ = True # source:False
_fields_ = [
('argsz', ctypes.c_int32),
('flags', ctypes.c_int32),
('count', ctypes.c_int32),
('details', struct_example_detail * 0),
('PADDING_0', ctypes.array_struct_example_detail, 98),
]
Which doesn't work:
Traceback (most recent call last): File "test.py", line 10, in class struct_example(ctypes.Structure): File "test.py", line 17, in struct_example ('PADDING_0', ctypes.array_struct_example_detail, 98), AttributeError: 'module' object has no attribute 'array_struct_example_detail'
By contrast, this worked in legacy ctypeslib, where h2xml
/xml2py
produced:
from ctypes import *
class example_detail(Structure):
pass
example_detail._fields_ = [
('first', c_int),
('last', c_int),
]
class example(Structure):
pass
example._fields_ = [
('argsz', c_int),
('flags', c_int),
('count', c_int),
('details', example_detail * 0),
]
just want to know whether this works for python 2.7 or not? if not is there is any other ctypeslib
for python 2.7
if the source code is faulty, the codegeneration takes place without warning.
So sometimes, record size are "faulty" because the standard says that a faulty record is of size 1.
File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 517, in STRUCT_DECL
return self._record_decl(cursor, typedesc.Structure, num)
File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 594, in _record_decl
members.append(self.FIELD_DECL(field))
File "/home/other/Compil/ctypeslib/ctypeslib/codegen/cursorhandler.py", line 868, in FIELD_DECL
raise ValueError("Field has no displayname")
ValueError: Field has no displayname
When using clang2py
on Ubuntu (clang 4.0) it is necessary to add default includes directory to the search path of clang. It could be noted somehow in the README file. It would also be handy if the script forward clangparser
warnings to stdout by default.
clang2py doesn't generate constants/fields for enums.
enum myEnum {
ONE,
TWO,
FOUR = 4
};
Generate with
clang2py t.h -o t.py -v --debug 2>&1
DEBUG:clangparser:ARCH sizes: long:c_int64 longdouble:c_long_double_t
DEBUG:clangparser:t.h:1: Found a ENUM_DECL|myEnum|myEnum
DEBUG:handler:get_unique_name: name "myEnum"
DEBUG:utils:ENUM_DECL: displayname:'myEnum'
DEBUG:handler:get_unique_name: name "myEnum"
DEBUG:clangparser:register: myEnum
DEBUG:codegen:Input was parsed
DEBUG:codegen:generate Enumeration, myEnum
###########################
# Symbols defined:
#
# Variables: 0
# Struct/Unions: 0
# Functions: 0
# Enums: 1
# Enum values: 0
# Typedefs: 0
# Pointertypes: 0
# Arraytypes: 0
# unknown functions: 0
# unknown variables: 0
#
# Total symbols: 1
###########################
needed 1 loop(s)
Output is:
# -*- coding: utf-8 -*-
#
# TARGET arch is: []
# WORD_SIZE is: 8
# POINTER_SIZE is: 8
# LONGDOUBLE_SIZE is: 16
#
import ctypes
# values for enumeration 'myEnum'
myEnum = ctypes.c_int # enum
__all__ = \
['myEnum']
Expected output something like this:
ONE = 0
TWO = 1
FOUR = 4
# values for enumeration 'myEnum'
myEnum = ctypes.c_int # enum
__all__ = \
['myEnum', 'ONE', 'TWO', 'FOUR']
It is not at all clear what the dependencies of clang2py are.
For example, clang2py fails with key error "packed" with libclang-6.0-dev on debian and pypi clang
Is it possible to avoid following exception?:
ctypeslib.codegen.handler.DuplicateDefinitionException: 'register: struct_mystruct_t already existed: struct_mystruct_t'
I am defining a name (myhandle) for an internal structure (mystruct_t), which is defined in an internal header file but hidden from API (opaque pointer):
typedef struct mystruct_t myhandle;
When this header file is included in multiple places and they are fed to clang2py, it stops with above exception. Is there a way to handle this without error?
def test_struct_named_twice(self):
ns = self.convert('''
typedef struct xyz {
int a;
} xyz;
''')
Consider the source and generated python below.
#define PRE "before"
#define POST " after"
#define PREPOST PRE POST
char a[] = "what";
char b[] = "why" " though";
char c[] = PRE POST;
char d[] = PREPOST;
PRE = "before" # macro
POST = "after" # macro
PREPOST = ['PRE', 'POST'] # macro
a = (ctypes.c_ubyte * 4).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'a')
b = (ctypes.c_ubyte * 3).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'b')
c = (ctypes.c_ubyte * 3).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'c')
d = (ctypes.c_ubyte * 7).in_dll(_libraries['/home/debian/sonicsd/test.so'], 'd')
The size for the char a[] is correct, but if initialized with multiple strings (as with char b[] ) it only gets the length of the first string. If it is initialized with a macro it (char c[]) it gets the length of the macro name instead of what it expands too. I have tried this both with and without linking in the shared lib.
Also note that the code generated for the PREPOST macro is not the same as what would be generated in h2xml/xml2py, which would expand the macro so that PREPOST = "why though"
in the generated python. I can see how this behavior may not always be desired, but it would be good to have as an option. This may deserve its own issue but it seemed related.
Could this have anything to do with the error messages clang2py outputs:
WARNING:handler:MACRO_INSTANTIATION is not handled
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/T1
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/PRE
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/POST
WARNING:handler:_do_nothing for MACRO_INSTANTIATION/PREPOST
This causes an error:
struct foo {
int bar;
};
typedef struct foo foo_t[256];
typedef struct {
foo_t baz;
} __somestruct;
File "ctypeslib/codegen/clangparser.py", line 1035, in FIELD_DECL
raise TypeError('Field can not be None %s'%(name ))
TypeError: Field can not be None baz
--clang-args CLANG_ARGS clang options
$ clang2py --clang-args -std=c99 test.h > /dev/null || echo failed
usage: clang2py [-h] [-c] [-d] [--debug] [-e] [-k TYPEKIND] [-i] [-l DLLS]
[-m module] [-o OUTPUT] [-p DLL] [-q] [-r EXPRESSION]
[-s SYMBOL] [-t TARGET] [-v] [-V] [-w W] [-x]
[--show-ids SHOWIDS] [--max-depth N] [--clang-args CLANG_ARGS]
files [files ...]
clang2py: error: argument --clang-args: expected one argument
failed
$ clang2py --clang-args=-std=c99 test.h > /dev/null && echo success
success
$ clang2py --clang-args=-std=c99 -Wall test.h > /dev/null || echo failed
usage: clang2py [-h] [-c] [-d] [--debug] [-e] [-k TYPEKIND] [-i] [-l DLLS]
[-m module] [-o OUTPUT] [-p DLL] [-q] [-r EXPRESSION]
[-s SYMBOL] [-t TARGET] [-v] [-V] [-w W] [-x]
[--show-ids SHOWIDS] [--max-depth N] [--clang-args CLANG_ARGS]
files [files ...]
clang2py: error: unrecognized arguments: -Wall
failed
$ clang2py --clang-args="-std=c99 -Wall" test.h > /dev/null && echo success
success
-l DLLS, --include-librarie DLLS libraries to search for exported functions
$ clang2py --include-librarie /lib/x86_64-linux-gnu/libc.so.6 test.h >/dev/null && echo success
success
$ clang2py --include-librarie /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libm.so.6 test.h > /dev/null || echo failed
Traceback (most recent call last):
File "/usr/local/bin/clang2py", line 6, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/clang2py.py", line 310, in main
flags=clang_opts)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/codegenerator.py", line 872, in generate_code
parser.parse(srcfile)
File "/usr/local/lib/python2.7/dist-packages/ctypeslib/codegen/clangparser.py", line 103, in parse
self.tu = index.parse(filename, self.flags, options=self.tu_options)
File "/usr/lib/python2.7/dist-packages/clang/cindex.py", line 2210, in parse
self)
File "/usr/lib/python2.7/dist-packages/clang/cindex.py", line 2322, in from_source
raise TranslationUnitLoadError("Error parsing translation unit.")
clang.cindex.TranslationUnitLoadError: Error parsing translation unit.
failed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.