pybind / pybind11 Goto Github PK
View Code? Open in Web Editor NEWSeamless operability between C++11 and Python
Home Page: https://pybind11.readthedocs.io/
License: Other
Seamless operability between C++11 and Python
Home Page: https://pybind11.readthedocs.io/
License: Other
Hi,
tried to build on OSX Yosemite,
but it seems something is wrong, example.so doesn't load properly on OSX
__Py_ZeroStruct
is not properly exported...
I'll have a look to see if I can figure out what is going on,
but if anybody has some idea what could be the issue much appreciated...
2016.01.02:19.42][Jiri@JiriMBP:~/lib/cpp/python/pybind11.orig/example]$ python
Python 3.4.3 |Anaconda 2.4.1 (x86_64)| (default, Oct 20 2015, 14:27:51)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so, 2): Symbol not found: __Py_ZeroStruct
Referenced from: /Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so
Expected in: flat namespace
in /Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so
>>>
System:
Darwin Kernel Version 14.5.0: Tue Sep 1 21:23:09 PDT 2015; root:xnu-2782.50.1~1/RELEASE_X86_64 x86_64 i386 MacBookPro11,2 Darwin
When using pybind11 from pypi what would be the best way to determine the
directory with include files, in a manner that would work regardless if
pybind11 have been installed system wide, using user scheme, virtualenv, etc?
Maybe you have some suggestion given that pybind11 is distributed on pypi.
I will note, that I like how numpy installation scheme works, which just places
headers along with the python module, making it easy to locate from python:
import numpy.core as core
d = os.path.join(os.path.dirname(core.__file__), 'include')
Though it looks quite differently in pybind11 package (ignoring the fact the
pybind11, doesn't provide any python module).
I'd like to create a Python object from a Python module I import within C++. The code would probably explain it best:
auto threading = py::module::import("threading");
py::object Thread = threading.attr("Thread");
py::function f = py::cast([]() { std::cout << "Hello" << std::endl; });
m.attr("t") = Thread.call(Py_None, f);
The first three lines work fine, but the last one doesn't.
The error I get (within Python) is:
terminate called after throwing an instance of 'pybind11::cast_error'
what(): handle::call(): unable to convert input arguments to Python objects
The equivalent Python code works fine:
import threading
def f(): print "Hello"
t = threading.Thread(None,f)
This is just a small one:
template class type_caster<holder_type>
should be
template <> class type_caster<holder_type>
Tested with VC2015
Hello,
This is not necessarily an issue, but a question. Sorry to ask here. I couldn't find an appropriate forum to ask questions about pybind11.
I've a python3 program and I want to rewrite some of the functionality in C++. The function I want to rewrite takes bytes
as input. What would be the correct signature for corresponding C++ function? I'm trying to use pybind11 for the binding.
Thanks.
I cannot compile the examples out of the latest commit (ad7bc01) on my Lubuntu 15.04 machine.
$ gcc --version
gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2
And the build log:
$ cmake .
-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Setting build type to 'MinSizeRel' as none was specified.
-- Found PythonLibs: /usr/lib/i386-linux-gnu/libpython2.7.so (found version "2.7.9")
-- Found PythonInterp: /usr/bin/python (found version "2.7.9")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pladow/pybind11
$ make -j 4
Scanning dependencies of target example
[ 7%] [ 15%] [ 23%] [ 30%] Building CXX object CMakeFiles/example.dir/example/example2.cpp.o
Building CXX object CMakeFiles/example.dir/example/example3.cpp.o
Building CXX object CMakeFiles/example.dir/example/example.cpp.o
Building CXX object CMakeFiles/example.dir/example/example1.cpp.o
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example3.cpp:10:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example1.cpp:11:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example.cpp:10:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example2.cpp:11:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
CMakeFiles/example.dir/build.make:54: recipe for target 'CMakeFiles/example.dir/example/example.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/example.dir/build.make:123: recipe for target 'CMakeFiles/example.dir/example/example3.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example3.cpp.o] Error 1
CMakeFiles/example.dir/build.make:77: recipe for target 'CMakeFiles/example.dir/example/example1.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example1.cpp.o] Error 1
CMakeFiles/example.dir/build.make:100: recipe for target 'CMakeFiles/example.dir/example/example2.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example2.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/example.dir/all' failed
make[1]: *** [CMakeFiles/example.dir/all] Error 2
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2
Hi
I am using pybind11 to wrap a lot of code, but hit the following problem.
A toy version of the problem is as follows.
Given the following code (hooked it up in the example directory of the pybind11 repo)
#include "example.h"
#include <pybind11/stl.h>
class ExampleJ1 {
public:
int n{-1};
ExampleJ1() = default;
void doSomething( int n_ ) { n = n_; }
};
void init_exJ1(py::module &m) {
py::class_<ExampleJ1>(m, "ExampleJ1", "Example J1 documentation")
.def(py::init<>())
.def("doSomething", &ExampleJ1::doSomething, "set the value n")
.def_readwrite("n", &ExampleJ1::n, "the value n")
;
}
i get the following error
>>> from example import ExampleJ1
>>> e = ExampleJ1
>>> e.n
<property object at 0x1021a5688>
>>> e.n = 1
>>> e.n
1
>>> e.doSomething(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Incompatible function arguments. The following argument types are supported:
1. (example.ExampleJ1, int) -> None
I am at loss, what I m doing wrong here.
Any idea why this doesn't work ?
Thanks
Jiri
master
as of 80a0674 currently generates warnings on MSVC:
Debug
mode: https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.141/job/c0yeltp07919w9dv https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.141/job/xfv0nem114pjq3fhNK : warning LNK4075: ignoring '/INCREMENTAL' due to '/LTCG' specification [C:\projects\pybind11\example.vcxproj]
Generating code
c:\program files (x86)\microsoft visual studio 14.0\vc\include\xatomic.h(1621): warning C4702: unreachable code [C:\projects\pybind11\example.vcxproj]
c:\program files (x86)\microsoft visual studio 14.0\vc\include\xatomic.h(1402): warning C4702: unreachable code [C:\projects\pybind11\example.vcxproj]
Finished generating code
I want to assign a class, exposed to Python through pybind11, to a module's attribute. This requires converting it to an object
. To make things concrete, I want to use RAII pattern to initialize and finalize MPI whenever the module is imported. The code looks roughly like this:
struct environment
{
environment() { MPI_Init(0,0); }
~environment() { MPI_Finalize(); }
};
py::class_<environment>(m, "environment")
.def("__repr__", [](const environment&) { return "MPI Environment"; });
m.attr("env") = py::detail::type_caster<environment>::cast(new environment, py::return_value_policy::automatic, m.ptr());
This works, but it's the last line that bothers me. Is this really the right way to do this? Have I missed some cleaner way? Besides being verbose, having to access detail
namespace seems wrong. I'm probably missing something, but if not, maybe it's worth adding clean support for this use-case?
I'm getting some errors when building the example with GCC 4.9 on linux. It looks like pybind::ssize_t
is actually int
on 32-bit, so overloading int_
with both on 32 bit architectures actually triggers an error.
Scanning dependencies of target example
[ 7%] Building CXX object CMakeFiles/example.dir/example/example.cpp.o
In file included from /pybind11/include/pybind11/cast.h:13:0,
from /pybind11/include/pybind11/pybind11.h:26,
from /pybind11/example/example.h:1,
from /pybind11/example/example.cpp:10:
/pybind11/pytypes.h:276:5: error: 'pybind11::int_::int_(pybind11::ssize_t)' cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/pybind11/include/pybind11/pytypes.h:273:5: error: with 'pybind11::int_::int_(int)'
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
CMakeFiles/example.dir/build.make:54: recipe for target 'CMakeFiles/example.dir/example/example.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/example.dir/all' failed
make[1]: *** [CMakeFiles/example.dir/all] Error 2
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2
The following works on Mac OS X, but not on Linux. (The obvious bits below, declarations of m
and such, are ommited.)
// common.h
struct A {};
// a.cpp
#include "common.h"
PYBIND11_PLUGIN(a)
{
py::class_<A>(m, "A")
.def(py::init<>());
}
// b.cpp
#include "common.h"
PYBIND11_PLUGIN(b)
{
m.def("test", [](A a) { fmt::print("test\n"); });
}
# test.py
import a
import b
x = a.A()
b.test(x)
On Mac, this works fine. On Linux, I get:
TypeError: Incompatible function arguments. The following argument types are supported:
1. (A) -> None
Hi,
First off, great job on the library. It looks very promising.
Now to my question. I have a set of functions where each one returns a custom container class called vec. Below is an example wrapper using boost.python with num_util by Phil Austin. Basically num_util takes an iterator and returns a numpy array.
How would I achieve the same results using pybind11?
namespace nbpl = num_util;
namespace bp = boost::python;
bpn::array mybartlett(int L)
{
// This is a call to my custom function that returns a custom vector
vec w = bartlett(L);
// This copies the data in w into the numpy array out
bpn::array out = nbpl::makeNum(w.begin(), w.size());
return out;
}
I just have this maybe sort of silly question. For what I understand this library can help us create python wrapper of c++ code easily, right? So the compiled module for python, converted from c++, will be really efficient, since the functioning part of the code is actually written by c++. Is my understanding correct? Thanks.
Currently if I want to access a parameter such as array length, the best method to my knowledge is
size_t size = target.request().count;
There are two problems with this. First, the corresponding python name is size
, so renaming count -> size
would be nice.
Further, wouldnt It be nice to have access to this as a free function?
size_t size = target.size()
list and dict are suported by pybind11 (just like by boost::python), but support for the set builtin type is missing (also from boost::python -- see http://thread.gmane.org/gmane.comp.python.c++/15129). Could set be supported by pybind11?
I expect this is a trivial question, and apologies for that, but I cannot work out how to do this.
If I have exposed a C++ type to python using pybind11 and created several instances of it in python, stored in a list, what is the best way to obtain these instances in a different C++ function (jn this case the constructor of another class). I tried passing the list directly to a function expecting a std::vector<MyType>
but this does not work. i.e.
py::class_<AnotherType>(m, "AnotherType")
.def("__init__",
[](AnotherType &instance, std::vector<MyType> instances) {
...
}
I also wrote a lambda function in C++ to accept a py::list
object; that worked, but I do not know how to extract the C++ types from the py::objects inside the list.
py::class_<AnotherType>(m, "AnotherType")
.def("__init__",
[](AnotherType &instance, py::list instances) {
for (auto inst : instances) {
//what do I do here?
}
...
}
What I actually want in the end is a std::vector<std::shared_ptr<MyType>>
.
What is the best way to achieve this?
Thx for the great project! ✨
During compile I get several warnings about:
./include/pybind11/attr.h(257)
./include/pybind11/attr.h(265)
./include/pybind11/attr.h(269)
./include/pybind11/pybind11.h(566): warning: offsetof applied to non-POD types is nonstandard
Tested with master as of 48548ea, nvcc/7.5.17
& gcc/4.9.2
.
This would be pretty easy to add, but would be quite nice to have by default (if not already present). Adding an RAII style object that releases the GIL when constructed and acquires the GIL when destructed. Having this object would make it easy to have blocks of C++ code that can run in parallel to the Python interpreter.
Suppose I want to process an py::dict
instance which might contain different types of key/value pairs, i.e. str/int
, str/dict
. After reading pybind11/pytypes.h
, I figure out that such problem could be solved by two steps:
handle::get_type
to get the type of instance.type_handle.ptr()
to Py[Type]_Check
methods to detect the type.I'm wondering if there is a better way to resolve the problem, such as function overloading or use some magic code to detect the type of of instance.
Thanks for your work, it's amazing!
When WITH_THREAD
is #undef
ed in $PYTHON_PREFIX/include/python2.7/pyconfig.h
.
./pybind11/include/pybind11/pybind11.h: In function ‘void pybind11::init_threading()’:
./pybind11/include/pybind11/pybind11.h:915:51: error: ‘PyEval_InitThreads’ was not declared in this scope
inline void init_threading() { PyEval_InitThreads(); }
This problem is also probably in python3.x
How I got here? 3rd party custom python and I cannot change it.
When trying to convert set types, I get a compilation error:
pybind11/include/pybind11/stl.h:87:37: error: invalid conversion from ‘long int’ to ‘PyObject* {aka _object*}’ [-fpermissive]
The source seems to be these lines (starting at line 87):
if (PySet_Add(set, value) != 0) {
Py_DECREF(value);
Py_DECREF(set);
return nullptr;
I think value
on lines 87 and 88 should be value_
, which is the PyObject * created on line 82:
PyObject *value_ = value_conv::cast(value, policy, parent);
Currently we implement decay here, but too me, we should be able to use std::decay. Any reason we don't?
I am trying to use pybind11 instead of boost python and I didn't found the way may to set the correct call policies for my function call. (Maybe I missed something in the doc)
I have an object which takes in its constructor, a list as argument and the lifetime of the list and the object need to be tied. In boost I need to use with_custodian_and_ward call policy.
What is the equivalent in pybind11 ?
It's not entirely wrong, but calling it the Zlib license in the readme would be a lot clearer.
In general type casters for types that hold multiple values are not exception
safe during conversion from C++ to Python objects, for example:
The same is true of casters for std::vector, std::set, std::map.
Why those caster would throw an exception in a first place? There are a few
places where explicit throw is used instead of null return value - I am not
sure if those are accidental or there is supposed to be distinction between
those two situations? In addition, memory allocation in copy_constructor may
throw, or C++ copy constructor itself may throw an exception.
In cases where operator new for a class is inaccessible, bound functions that return that type with the return_value_policy::reference
will not compile
#include <memory>
#include <iostream>
#include <pybind11/pybind11.h>
using namespace pybind11;
class Foo {
void *operator new(size_t bytes) throw();
};
Foo gFoo;
Foo& getstmt() {
return gFoo;
}
PYBIND11_PLUGIN(example) {
module m("example");
class_<Foo>(m, "Foo");
m.def("getstmt", getstmt, return_value_policy::reference);
return m.ptr();
}
generating errors in type_caster_generic::cast
/usr/local/include/pybind11/cast.h:214:16: error: 'operator new' is a private member of 'Foo'
return new type(*((const type *) arg));
^~~
/usr/local/include/pybind11/cast.h:202:80: note: in instantiation of function template specialization
'pybind11::detail::type_caster<Foo, void>::copy_constructor<Foo, 0>' requested here
return type_caster_generic::cast(&src, policy, parent, &typeid(type), ©_constructor);
^
/usr/local/include/pybind11/pybind11.h:66:39: note: in instantiation of member function
'pybind11::detail::type_caster<Foo, void>::cast' requested here
handle result = cast_out::cast(
^
/usr/local/include/pybind11/pybind11.h:469:22: note: in instantiation of function template
specialization 'pybind11::cpp_function::cpp_function<Foo &, pybind11::name, pybind11::sibling,
pybind11::return_value_policy>' requested here
cpp_function func(std::forward<Func>(f), name(name_),
^
main.cpp:21:7: note: in instantiation of function template specialization 'pybind11::module::def<Foo
&(&)(), pybind11::return_value_policy>' requested here
m.def("getstmt", getstmt, return_value_policy::reference);
^
main.cpp:8:11: note: implicitly declared private here
void *operator new(size_t bytes) throw();
The equivalent boost::python example does compile.
#include <memory>
#include <boost/python.hpp>
using namespace boost;
using namespace boost::python;
class Foo {
void *operator new(size_t bytes) throw();
};
Foo gFoo;
Foo& getstmt() {
return gFoo;
}
BOOST_PYTHON_MODULE(example)
{
class_<Foo>("Foo");
def("getstmt", getstmt, return_value_policy<reference_existing_object>());
}
The header pybind11/numpy.h
includes complex.h
which, at least in glibc, defines I
as a preprocessor macro to denote the imaginary unit. Unfortunately I
is a popular name for template parameters in e.g. the boost library. Therefore, the following two lines will fail to compile
#include <pybind11/numpy.h>
#include <boost/variant.hpp>
while the following three lines will work fine.
#include <pybind11/numpy.h>
#undef I
#include <boost/variant.hpp>
The compiler error is very long, however it starts with
include/boost/move/iterator.hpp:151:11: error: declaration of anonymous class must be a definition
template <class I>
which indicates that the I
is the problem.
It took me some time to find the cause for that particular error. Do you think it would be reasonable to #undef I
within pybind11/numpy.h
to spare others from the same problem?
A common use-case for my python extensions is to do some calculations that are difficult to vectorize using numpy. They usually accept some numpy arrays and will also return one but can't be simplified so that py::vectorize can be used. I've found that the documentation could use an example on how to do this. From reading the implementation of py::vectorize in numpy.h, I've managed to assemble the following:
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
namespace py = pybind11;
py::array_t<double> add_arrays(py::array_t<double> i, py::array_t<double> j)
{
py::buffer_info info_i = i.request();
py::buffer_info info_j = j.request();
if ((info_i.ndim != 1) || (info_j.ndim != 1))
throw std::runtime_error("Number of dimensions must be one");
if (info_i.shape[0] != info_j.shape[0])
throw std::runtime_error("Input shapes must be equal");
std::vector<double> ret(info_i.shape[0]);
for (unsigned int idx = 0; idx < info_i.shape[0]; idx++)
ret[idx] = ((double*)info_i.ptr)[idx] + ((double*)info_j.ptr)[idx];
std::vector<size_t> strides = {sizeof(double)};
std::vector<size_t> shape = {info_i.shape[0]};
size_t ndim = 1;
return py::array(py::buffer_info(ret.data(), sizeof(double),
py::format_descriptor<double>::value(),
ndim, info_i.shape, strides));
}
PYBIND11_PLUGIN(test)
{
py::module m("test", "Testing pybind11 with numpy arrays");
m.def("add_arrays", &add_arrays, "Adding two numpy arrays");
return m.ptr();
}
Furthermore I've found that for 1D-arrays, the return line could be further simplified to
return py::array(info_i.shape[0], ret.data());
I wonder if this implementation is good or if things could be improved. If I understand correctly, during construction of the numpy array, the memory from the vector is copied. Is it possible to prevent that copy operation somehow?
However for the actual request I think this kind of code would be very interesting to people transitioning from Matlab to Python who want to have a replacement for their MEX extensions. This kind of task (non-vectorizable function on arrays with a returned array) is pretty common there and I spent quite some time figuring it out. While it taught me some interesting things, a good example might save other people from having to do that.
If the above code is good I can also provide a pull request for the docs based on it.
Hi Jakob,
In C++ I have a class Space
with public member vector<PointParticle> p
and a non-class function template,
template<class Tspace>
void cm2origo(Tspace::Tparticlevector &p) { /* modify p */ }
Wrapping is done with,
typedef Space<PointParticle> Tspace;
m.def("cm2origo", &cm2origo<Tspace>);
py::class_<Tspace>(m, "Space")
.def_readwrite("p", &Tspace::p)
.def ...
In python I can read/write to p
as one would expect. However, when calling the exposed cm2origo
function, changes to p
are not transferred back to python. Any suggestions how to sort this out?
I'm getting a compilation error when using a unary operator. Here's some sample code that exhibits the problem.
This is on Lubuntu 15.04 with gcc 4.9.2. Also on Kubuntu 14.04 with gcc 4.8.4.
#include <pybind11/pybind11.h>
#include <pybind11/operators.h>
namespace py = pybind11;
class foo
{
public:
foo(int f_) : m_f(f_)
{
}
foo operator~() const
{
return foo(~m_f);
}
private:
int m_f;
};
PYBIND11_PLUGIN(foo)
{
py::module m("foom", "foom example");
py::class_<foo>(m, "foo")
.def(py::init<int>())
.def(~py::self);
return m.ptr();
}
I get the following errors:
In file included from test.cpp:2:0:
./pybind11/operators.h: In instantiation of ‘struct pybind11::detail::op_impl<(pybind11::detail::op_id)15, (pybind11::detail::op_type)2, foo, foo, pybind11::detail::undefined_t>’:
./pybind11/operators.h:52:29: required from ‘void pybind11::detail::op_<id, ot, L, R>::execute(pybind11::class_<Base, Holder>&, Extra&& ...) const [with Base = foo; Holder = std::unique_ptr<foo, std::default_delete >; Extra = {}; pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t]’
./pybind11/pybind11.h:703:9: required from ‘pybind11::class_<type, holder_type>& pybind11::class_<type, holder_type>::def(const pybind11::detail::op_<id, ot, L, R>&, Extra&& ...) [with pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t; Extra = {}; type = foo; holder_type = std::unique_ptr<foo, std::default_delete >]’
test.cpp:29:19: required from here
./pybind11/operators.h:96:17: error: ‘l’ was not declared in this scope
static auto execute(const L &l) -> decltype(expr) { return expr; }
^
./pybind11/operators.h:137:1: note: in expansion of macro ‘PYBIND11_UNARY_OPERATOR’
PYBIND11_UNARY_OPERATOR(invert, operator~, ~l)
^
./pybind11/operators.h: In instantiation of ‘void pybind11::detail::op_<id, ot, L, R>::execute(pybind11::class_<Base, Holder>&, Extra&& ...) const [with Base = foo; Holder = std::unique_ptr<foo, std::default_delete >; Extra = {}; pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t]’:
./pybind11/pybind11.h:703:9: required from ‘pybind11::class_<type, holder_type>& pybind11::class_<type, holder_type>::def(const pybind11::detail::op_<id, ot, L, R>&, Extra&& ...) [with pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t; Extra = {}; type = foo; holder_type = std::unique_ptr<foo, std::default_delete >]’
test.cpp:29:19: required from here
./pybind11/operators.h:52:32: error: ‘execute’ is not a member of ‘op {aka pybind11::detail::op_impl<(pybind11::detail::op_id)15, (pybind11::detail::op_type)2, foo, foo, pybind11::detail::undefined_t>}’
class_.def(op::name(), &op::execute, std::forward(extra)...);
^
I noticed that including pybind11/functional.h
has some (probably) unintended side-effects.
If I include pybind11/functional.h
I cannot pass std::function
instances which I get from C++ code back to C++ functions. On the other hand without pybind11/functional.h
, I cannot pass Python methods to C++ functions expecting std::function
.
Is it possible to get both working, or are they mutually exclusive? I'm attaching a short example
#include <pybind11/pybind11.h>
#include <pybind11/functional.h> // comment/uncomment this
struct Functor {
bool operator() (int a, int b) const {
return a < b;
}
std::function<bool(int, int)> cast() {
return std::function<bool(int, int)>(*this);
}
};
struct Caller {
bool call(std::function<bool(int, int)> func, int a, int b) {
return func(a, b);
}
};
namespace py = pybind11;
PYBIND11_PLUGIN(tmp) {
py::module m("tmp", "pybind11 tmp plugin");
py::class_<Caller>(m, "Caller").def(py::init<>())
.def("call", &Caller::call);
py::class_<std::function<bool(int, int)>>(m, "FuncType").def(py::init<>());
py::class_<Functor>(m, "Functor").def(py::init<>())
.def("__call__", [](const Functor& f, int a, int b) { return f(a, b); })
.def("cast", &Functor::cast);
return m.ptr();
}
import tmp
cppfunc = tmp.Functor()
caller = tmp.Caller()
def pyfunc(a, b):
return a < b
try:
print 'Calling C++ functor {0!r}'.format(cppfunc),
res = cppfunc(1, 2)
print 'OK'
except Exception as error:
print 'FAILED with error {1!r}'.format(cppfunc, error)
try:
print 'Calling C++ with Python function {0!r}'.format(pyfunc),
caller.call(pyfunc, 1, 2)
print 'OK'
except Exception as error:
print 'FAILED with error {1!r}'.format(cppfunc, error)
stdfunc = cppfunc.cast()
try:
print 'Calling C++ with C++ function {0!r}'.format(stdfunc),
caller.call(stdfunc, 1, 2)
print 'OK'
except Exception as error:
print 'FAILED with error {1!r}'.format(cppfunc, error)
Without pybind11/functional.h
Calling C++ functor <tmp.Functor object> OK
Calling C++ with Python function <function pyfunc> FAILED with error TypeError('Incompatible function arguments. The following argument types are supported:\n 1. (Caller, std::function<bool (int, int)>, int32_t, int32_t) -> bool\n',)
Calling C++ with C++ function <tmp.FuncType object> OK
With pybind11/functional.h
Calling C++ functor <tmp.Functor object> OK
Calling C++ with Python function <function pyfunc> OK
Calling C++ with C++ function <built-in method of PyCapsule object> FAILED with error TypeError('Incompatible function arguments. The following argument types are supported:\n 1. (Caller, function<(int32_t, int32_t) -> bool>, int32_t, int32_t) -> bool\n',)
Hi,
I am looking for a way to create python plugins for my c++ applications. That is, load a user-defined .py file and run a python function in it with arguments from my running C++ application.
Can I do that with pybind11?
I clone master of pybind11
cmake .
make -j8
I got errors
http://pastebin.com/TrXpEwqL
gcc version 5.3.0 (GCC)
python 3.5m
What information do you need?
Thank
I'm not sure if I just can't find it, but it would be great to have the analog of boost::python::import
to import a module as a pybind::object
, and then to get its functions via attr
.
The automatic transparent conversions do not seem to work for std::shared_ptr
.
I looked at the example8 and it seems that ref<T>
class there differs from shared_ptr
in one apparently crucial aspect, it has user-defined conversion
operator T* () { return m_ptr; }
which allows it to be passed as the first argument to cast
function.
I managed to get automatic conversion to work with shared_ptr
by adding
diff --git a/include/pybind11/cast.h b/include/pybind11/cast.h
index 5fe9342..1cb4ff4 100644
--- a/include/pybind11/cast.h
+++ b/include/pybind11/cast.h
@@ -527,6 +527,12 @@ public:
explicit operator type&() { return *(this->value); }
explicit operator holder_type&() { return holder; }
explicit operator holder_type*() { return &holder; }
+
+ using type_caster<type>::cast;
+ static PyObject *cast(const holder_type &src, return_value_policy policy, PyObject *parent) {
+ return type_caster<type>::cast(src.get(), policy, parent);
+ }
+
protected:
holder_type holder;
};
But perhaps there is a better way?
While trying to add icc
support I stumbled upon several lines defining methods such as
PYBIND11_NOINLINE inline [...]
which makes probably no sense.
Is that intentionally?
Probably two individual PYBIND11_NOINLINE
and PYBIND11_INLINE
macros would be more useful (using always only once of them when trying to force to (prevent) inlining).
In array.API
, several (but not all) numpy macros are declared as functions, this is all fine, but it causes issues when you import numpy before pybind11.
For example
#include <numpy/arrayobject.h>
#include <pybind11/pybind11.h>
causes compilation crashes because the macros (defined in numpy) interfere with the names defined in pybind11.
Hi,
I noticed the following compile error:
#include "example.h"
#include <pybind11/stl.h>
#include <vector>
using VecN = std::vector<size_t>;
struct ExampleJ1 {
VecN nn;
ExampleJ1( const VecN& nn = VecN{} ) : nn{nn} {}
};
void init_exJ1(py::module &m) {
py::class_<ExampleJ1>(m, "ExampleJ1", "Example J1 documentation")
.def(py::init<VecN>(), py::arg("nn") = VecN() )
.def_readwrite("nn", &ExampleJ1::nn, "the values nn");
}
gives
[ 6%] Building CXX object CMakeFiles/example.dir/example/exampleJ1.cpp.o
In file included from /Users/Jiri/lib/cpp/python/pybind11/example/exampleJ1.cpp:11:
In file included from /Users/Jiri/lib/cpp/python/pybind11/example/example.h:1:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pybind11.h:26:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/cast.h:13:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pytypes.h:12:
/Users/Jiri/lib/cpp/python/pybind11/include/pybind11/common.h:202:36: error: no matching function for call to 'to_string'
to_string(const T &value) { return std::to_string(value); }
^~~~~~~~~~~~~~
/Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pybind11.h:128:49: note: in instantiation of function template specialization
'pybind11::detail::to_string<std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > >' requested here
def[entry->keywords++] = strdup(detail::to_string(a.value).c_str());
Adding a specialization for container types to to_string would alleviate the problem.
What would you prefer to do here ?
Regards
Jiri
I'm quite new to C programming in general so I might be doing something wrong:
I downloaded the zip, and unzipped it and cded into the resulting dictionary.
Run the following commands:
cmake .
make -j 4
make test
The first two completed what appears to be successfully (no mention of errors). But make test says 12 out of 12 test failed and gives the following error:
Errors while running CTest
make: *** [test] Error 8
I'm running OS X 10.11.1 and $c++ --version
gives:
Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix
Additionally trying the example from the docs gives a massive error:
$c++ -O3 -shared -std=c++11 -I ~/Desktop/pybind11-master/include `python-config --cflags --libs` example.cpp -o example.so
In file included from example.cpp:1:
In file included from /Users/mik/Desktop/pybind11-master/include/pybind11/pybind11.h:26:
In file included from /Users/mik/Desktop/pybind11-master/include/pybind11/cast.h:13:
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:14: error: call to constructor of 'pybind11::int_' is ambiguous
int_ start(start_), stop(stop_), step(step_);
^ ~~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:29: error: call to constructor of 'pybind11::int_' is ambiguous
int_ start(start_), stop(stop_), step(step_);
^ ~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:42: error: call to constructor of 'pybind11::int_' is ambiguous
int_ start(start_), stop(stop_), step(step_);
^ ~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
3 errors generated.```
I have a new set of problems with integer conversions. It's not clear to me what's going on (and it requires another library to reproduce), but hopefully it will make sense to you.
You need mpi4py
for this example.
Define a function through pybind11:
m.def("print_comm", [](int x) { std::cout << "Got comm: " << x << std::endl; });
Now test it from Python:
from mpi4py import MPI
w = MPI.COMM_WORLD
a = MPI._addressof(w)
print type(a)
# prints: <type 'int'>
print_comm(a)
The last command produces:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-7-16f94faa6020> in <module>()
----> 1 h.print_comm(a)
TypeError: Incompatible function arguments. The following argument types are supported:
1. (int) -> None
I'm not sure how to debug this. If you have advice or any idea what's going on, I'd appreciate it.
Boost::python supports, and it would be nice if pybind11 did aswell, conversion of errors. I have legacy code which contains this snippet
if (index < 0 || index >= size())
throw std::out_of_range("index out of range");
in boost python, this was converted to IndexError
in python. This allows my container to be used with python expressions such as list(my_object)
since this is implemented something like this in python
def list(obj):
result = []
i = 0
while True:
try:
result += [obj[i]]
i += 1
except IndexError:
break
But with pybind11, it becomes a RunTimeError
and all such logic fails.
I came across this project, yesterday, and I'd like to say, I really like the project, improving on Boost.Python's environment and API quirks, and I'd like to use it in my C++-to-Python wrapper.
But as I was looking, yesterday, I came across a problem. I was trying to expose a leaf class of a large hierarchy, and encountered a problem when calling a method of the base class that wasn't overridden in the exposed class.
Here is the stripped down example of what I was trying to do:
#include <iostream>
class A
{
public:
A() : x(1) {}
virtual void f() { std::cout << "A::f" << std::endl; }
virtual void g() { std::cout << "A::g" << std::endl; }
void h() { std::cout << "A::h" << std::endl; }
void i() { std::cout << "A::i" << std::endl; }
int x;
};
class B : public A
{
public:
virtual void f() { std::cout << "B::f" << std::endl; }
void h() { std::cout << "B::h" << std::endl; }
};
#include "pybind11/pybind11.h"
namespace py = pybind11;
PYBIND11_PLUGIN(test)
{
py::module m("test");
// py::class_<A> a(m, "A");
py::class_<B>(m, "B"/*, a*/)
.def(py::init<>())
.def("f", &B::f)
.def("g", &B::g)
.def("h", &B::h)
.def("i", &B::i)
.def_readwrite("x", &B::x)
;
}
If I then import the resulting library in Python, and try to use it, I get the following:
>>> import test
>>> b = test.B()
>>> b.i()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Incompatible function arguments. The following argument types are supported:
1. (A) -> None
The same goes for calling 'g' and accessing 'x'.
I tried the same with Boost.Python, and it does not seem to have this problem. (Which is a pitty, 'cause again, I really like this slim header-only library, instead of Boost ;-) )
EDIT: Oh, yes, almost forgot. If you remove the 2 comments, and expose the A class and the fact that is is B's base class, everything works as it should.
I have a C++ function which returns a enumerated type.
However, when I compare the returned value with another value the results is always false.
This happend in example4.cpp if you had:
EMyEnumeration returnEnum(){
return EFirstEntry;
}
...
m.def("returnEnum", &returnEnum);
The following python code returns False each time:
import example
example.returnEnum() == example.EFirstEntry
example.returnEnum() == example.ESecondEntry
I think, the "eq" method is not defined and python compares address of the objects instead of values.
I was wondering about your thoughts on having def_property
(and friends) take a fdel
parameter to be called when the property is deleted.
Cython provides lightweight conversions between Foo *
and a numpy structured array with a corresponding dtype, where Foo
is a plain struct. E.g.,
struct Foo {
uint16_t a;
double b;
}
mapping to np.dtype([('a', '<u2'), ('b', '<f8')])
.
How would you deal with structured arrays in pybind11
?
When trying to move from boost::python to pybind11 (I really love this initiative), i learned that pybind11 does not seem to be properly const-correct. This causes issues, for example, one would expect this to work:
void my_fun(const py::array& arr) {
size_t size = arr.request().count
}
It fails because request is not declared const
. The same applies to many other functions.
-flto
option prevents the example code from building on Linux with Clang (3.7). The problem is with the linker, it simply complains:
CMakeFiles/example.dir/example/example.cpp.o: file not recognized: File format not recognized
Removing -flto
fixes everything. Apparently, this is not an unusual issue with Clang.
Can we add an option that shows you what code it actually put to compile against?
In a project where I am using pybind11 I had a problem with C++ objects wrapped in shared_ptr that were handed over to Python and then re-entered C++. With default approach (using PYBIND11_DECLARE_HOLDER_TYPE) this results in two smart pointers that do not know about each other. Using std::enable_shared_from_this template in my objects I have modified the code generated by PYBIND11_DECLARE_HOLDER_TYPE to something like (basically there is only one small difference at line 7):
template <class T> class type_caster<std::shared_ptr<T>> : public type_caster_holder<T, std::shared_ptr<T>> {
typedef type_caster<T> parent;
public:
bool load(PyObject *src, bool convert) {
if (!parent::load(src, convert))
return false;
holder = ((T *) parent::value)->shared_from_this();
return true;
}
explicit operator T*() { return this->value; }
explicit operator T&() { return *(this->value); }
explicit operator std::shared_ptr<T>&() { return holder; }
explicit operator std::shared_ptr<T>*() { return &holder; }
using type_caster<T>::cast;
static PyObject *cast(const std::shared_ptr<T> &src, return_value_policy policy, PyObject *parent) {
return type_caster<T>::cast(src.get(), policy, parent);
}
protected:
std::shared_ptr<T> holder;
};
I hope that this will save some time to anyone who will have the same scenario. If there is a good way to support this by default in some way it would be also very nice. I am not that familiar with C++ templates, but I guess the template could be limited to classes that inherit from std::enable_shared_from_this.
Hi @wjakob,
I noticed that function overloading does not seems to work when keyword argument is used (or works in rather confusing way). Supplying at least one keyword argument (even when no default values is specified) lead to wrong function resolution. Pybind version used: somewhat latest from GitHub (1546b85).
Is this behavior expected? Or something wrong with my code?
Thanks,
Code to replicate this:
#include <pybind11/pybind11.h>
#include <string>
std::string foo(int a) { return "foo1"; }
std::string foo(int a, int b) { return "foo2"; }
std::string foo(int a, int b, int c) { return "foo3"; }
PYBIND11_PLUGIN(example) {
pybind11::module m("example", "example module");
m.def("foo", (std::string (*)(int) ) &foo, "doc", pybind11::arg("a"));
m.def("foo", (std::string (*)(int, int) ) &foo, "doc", pybind11::arg("a"), pybind11::arg("b"));
m.def("foo", (std::string (*)(int, int, int) ) &foo, "doc", pybind11::arg("a"), pybind11::arg("b"), pybind11::arg("c"));
return m.ptr();
}
Output:
In [1]: foo(1)
Out[1]: u'foo1'
In [2]: foo(1,2)
Out[2]: u'foo2'
In [3]: foo(1,2,3)
Out[3]: u'foo3'
In [4]: foo(a=1,b=2)
Out[4]: u'foo1'
In [5]: foo(a=1,b=2,c=3)
Out[5]: u'foo1'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.