Comments (9)
Users seem to expect binary wheels, not just sdist
-only releases, for PyPI packages with compiled code. I’ve seen a few options for building binary wheels for assorted platforms. I haven’t deployed any of them myself. It seems like cibuildwheel is a popular and relatively polished option. As is typical in Python packaging, there are almost as many workflows, processes, and combinations of tools as there are projects.
Integrating something like this into the release process will probably take some effort and decision-making by the project owners.
from rtree.
Comparison with selected other tools for wrapping C APIs:
- cffi is pleasantly simple and lightweight, but manual definition of function prototypes is still required, and mismatches are still possible. It is only suitable for wrapping C APIs; in this case, that would be fine. It’s mostly aimed at low-level bindings, akin to the current
ctypes
binding, and it’s usually possible to implement these with the interface definitions and little or no additional C code. - Cython offers a powerful general-purpose almost-Python language that is transpiled to C or C++. It can be particularly useful for high-level bindings that need to implement significant functionality in compiled code for performance reasons, especially when a goal is to minimize the amount of C or C++ code needed. Unlike
cffi
, it can wrap C++. Manual definition of function prototypes is still required, and mismatches are still possible. Updates to Cython aren’t always entirely backwards-compatible, and its language isn’t formally specified as C or C++ are. - Boost.Python is pretty similar to
pybind11
in concept, but it’s a heavier-weight dependency, is not header-only, and seems to have a much smaller user base today than other options in this list. - SWIG is still around, but—without going into detail—I think newer tools provide a better experience and produce better bindings. One mostly only sees it used for projects that have been around since SWIG and
ctypes
were the only mature choices and also have huge APIs.
For this particular project, in a quick up-front comparison, pybind11
still looks like a good choice.
from rtree.
Does pybind11 have mypy-compatible type hints? That was another problem I encountered with the current ctypes code.
from rtree.
Does pybind11 have mypy-compatible type hints? That was another problem I encountered with the current ctypes code.
It’s an important question. I can’t answer from experience. I’ve use ctypes, Cython, and CFFI in the past, but not pybind11—and all before the widespread use of typing in Python.
A quick search turns up mypy.stubgen for generating draft stubs for arbitrary C extensions, with the expectation that they will be edited and maintained by hand. There is also a third-party pybind11-stubgen (discussion).
I don’t see an obvious way to avoid some manual type hinting as @hobu suggested, but I could easily be missing something.
from rtree.
An open question is, if type hints indeed need to be manually maintained, is it worth the switch, and particularly, worth introducing compiled code to the package?
Using pybind11
would still have the advantage that most API mismatches with libspatialite
should produce compiler warnings or errors. (It might still be possible to have silent mismatches in some by-value parameters—I’m not sure.)
On the other hand, publishing binary wheels is a lot more annoying.
from rtree.
We are already in the business of publishing binary wheels. Doing this with ctypes+libspatialindex is just as annoying as it would be with pybind11+libspatialindex.
I would have expected some type hinting stuff coming out of pybind11 because the point of it is to have type checking when compiling wrappers 😄, but I can understand if it hasn't matured that far at this time.
The case for pybind11 is indeed to do away with the manually maintained and matched API calls of ctypes <-> libspatialindex to avoid the discrepancies of things like @musicinmybrain found in #222.
from rtree.
Yes, manual type hints are fine. It's just that with ctypes, it didn't seem to be possible to use manual type hints. Mypy forbids assignment of methods to override other instance methods. Also, the typeshed hints for ctypes seem to have the wrong number of arguments, so there were all kinds of issues. As long as pybind11 type hints are slightly simpler, I'll be happy with the change!
from rtree.
We are already in the business of publishing binary wheels. Doing this with ctypes+libspatialindex is just as annoying as it would be with pybind11+libspatialindex.
That’s right, I forgot about the bundled libspatialite
in the PyPI wheels. (Since Fedora’s RPM packaging uses the system copy of the library, it applies a small downstream patch that makes its python-Rtree
package pure-Python/noarch
.)
Then the conversion is probably worth attempting even if the gain is somewhat less.
If pybind11 isn’t offering automatic type hinting, it might mean that cffi
could also be a good solution, since when used in “API mode” it still includes the library’s C header and explicitly attempts to check for function signature mismatches.
It might just come down to which tool’s style you like better.
from rtree.
We don't have to migrate. Having done both pybind11 and ctypes, I like pybind11 better nowadays for the reasons I've stated. This codebase has been quite stable over the years, so I do not wish to upset that with a big subsystem change that has the potential for breakage for very little external benefit.
A case for pybind11 is the opportunity to ditch the C API usage of libspatialindex for direct consumption of its C++ API. That has the potential to speed things up and make code simpler and easier to follow.
The plan for now is to make the 1.0.0 release be based on ctypes to include all of the recent activity, and then we will evaluate pybind11 options if you or someone else has time to put in the effort.
from rtree.
Related Issues (20)
- Comparison in test_index.py::IndexSerialization::test_interleaving sometimes fails HOT 1
- Mismatch between package and New M1 apple silicon HOT 1
- rtree import still leads to dlsym Attribute Error and crashes geopandas HOT 1
- Python 3.10 wheels HOT 2
- rtree fails to import HOT 1
- ArgumentError: expected LP_LP_c_long instance instead of pointer to LP_c_long HOT 8
- typing_extensions now required HOT 1
- Using rtree in Cygwin HOT 1
- import rtree leads to `OSError: Could not load libspatialindex_c library` on aarch64 HOT 6
- Please release a new version with the type hints support HOT 1
- Index is checking write access despite assuming read-only access HOT 3
- why .dat file modified after intersection query?
- Nearest Neighbour Metric
- how to output more information about the constructed Rtree HOT 1
- QUESTION: How can rtree.index.Index properties be used to tweak performance ? HOT 2
- Linux binary wheels have two copies of libspatialindex HOT 1
- Calculating the Height of an R-Tree: Where Did I Go Wrong? HOT 1
- Issue with Python 3.12.0rc1 HOT 5
- "Unable to open file '%s' for index storage" - Write permissions on index should not be required HOT 1
- About RTree Data Structure Persistent Storage HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rtree.