ssciwr / pybind11-numpy-example Goto Github PK

View Code? Open in Web Editor NEW

23.0 3.0 8.0 319 KB

Example of using pybind11 with numpy and publishing to PyPI and conda-forge

License: MIT License

CMake 23.42% C++ 30.26% Python 31.29% Shell 15.03%

pybind11 cibuildwheel pypi cpp wheels conda-forge continuous-deployment continuous-integration python

pybind11-numpy-example's Introduction

pybind11-numpy-example

What

A simple example of how to use pybind11 with numpy and publish this as a library on PyPI and conda-forge.

This C++/Python library creates a std::vector of 16-bit ints, and provides a Python interface to the contents of this vector in a few different ways:

a Python List (copy the data)
a NumPy ndarray (copy the data).
a NumPy ndarray (move the data).

Why

Python Lists are great! However, when storing many small elements of the same type, a Numpy array is much faster and uses a lot less memory:

How

The pybind11 code is in src/pybind11_numpy_example_python.cpp.

The python package is defined in pyproject.toml and uses scikit-build-core.

Each tagged commit triggers a GitHub action job which uses cibuildwheel to build and upload a new release including binary wheels for all platforms to PyPI.

The conda-forge package is generated from this recipe, and automatically updates when a new version is uploaded to PyPI.

The scripts used to generate the above plots are in scripts.

This repo was quickly set up using the SSC C++ Project Cookiecutter.

pybind11-numpy-example's People

Contributors

Stargazers

Watchers

Forkers

stpstp henryiii peterimpetus roy-kid lunarlanding cubao lz0211 tienviitta

pybind11-numpy-example's Issues

add ndarray example without copying data

see pybind/pybind11#1042 (comment)

conda-forge package

This repo currently serves as a simple minimal example of how to build & release binary wheels for a python / pybind11 project on PyPI.

It would be nice to also publish it on conda-forge to show how this works.

install TARGETS given target "pybind11-numpy-example" which does not exist in this directory

Hello, when I try to compile this example with python setup.py install, the following error occurs, does someone know how to fix this?

running install
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing pybind11_numpy_example.egg-info/PKG-INFO
writing dependency_links to pybind11_numpy_example.egg-info/dependency_links.txt
writing top-level names to pybind11_numpy_example.egg-info/top_level.txt
reading manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
-- pybind11 v2.10.0 
CMake Error at CMakeLists.txt:52 (install):
  install TARGETS given target "pybind11-numpy-example" which does not exist
  in this directory.


-- The following REQUIRED packages have been found:

 * PythonInterp (required version >= 3.6)
 * PythonLibsNew

-- Configuring incomplete, errors occurred!
See also "/data/code_zxt/pybind11-numpy-example/build/temp.linux-x86_64-cpython-37/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "setup.py", line 101, in <module>
    "License :: OSI Approved :: MIT License",
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "setup.py", line 27, in run
    self.build_extension(ext)
  File "setup.py", line 62, in build_extension
    ["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/data/code_zxt/pybind11-numpy-example', '-DBUILD_DOCS=OFF', '-DBUILD_TESTING=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/code_zxt/pybind11-numpy-example/build/lib.linux-x86_64-cpython-37/', '-DPYTHON_EXECUTABLE=/home/zxt/anaconda3/envs/seg2/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
(seg2) zxt@octopus-A003:/data/code_zxt/pybind11-numpy-example$ clear
(seg2) zxt@octopus-A003:/data/code_zxt/pybind11-numpy-example$ python setup.py install
running install
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing pybind11_numpy_example.egg-info/PKG-INFO
writing dependency_links to pybind11_numpy_example.egg-info/dependency_links.txt
writing top-level names to pybind11_numpy_example.egg-info/top_level.txt
reading manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
-- pybind11 v2.10.0 
CMake Error at CMakeLists.txt:52 (install):
  install TARGETS given target "pybind11-numpy-example" which does not exist
  in this directory.


-- The following REQUIRED packages have been found:

 * PythonInterp (required version >= 3.6)
 * PythonLibsNew

-- Configuring incomplete, errors occurred!
See also "/data/code_zxt/pybind11-numpy-example/build/temp.linux-x86_64-cpython-37/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "setup.py", line 101, in <module>
    "License :: OSI Approved :: MIT License",
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "setup.py", line 27, in run
    self.build_extension(ext)
  File "setup.py", line 62, in build_extension
    ["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/data/code_zxt/pybind11-numpy-example', '-DBUILD_DOCS=OFF', '-DBUILD_TESTING=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/code_zxt/pybind11-numpy-example/build/lib.linux-x86_64-cpython-37/', '-DPYTHON_EXECUTABLE=/home/zxt/anaconda3/envs/seg2/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.

python wheels contain c++ library, headers and cmake config

The C++ headers, library and cmake config are incorrectly being included in the python wheels (thanks @carterbox)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.