Comments (32)
Hi @joerick. No worries, I'm happy to go through this. From my perspective, you're one of the most responsive open source developers I've ever had the pleasure to work with! I think this project will be super helpful to me as well as many others, and I'm happy to help out ironing the kinks =)
from cibuildwheel.
Turns out it was a weird bug related to setup_requires. Which is now fixed. Let me know how you get on!
pip install git+https://github.com/joerick/cibuildwheel@c3565d5025566dcf4b795eef8d40f38a12872151
from cibuildwheel.
@joerick Looks like it's all working now! Just need to figure out how to automagically put the wheels on pypi.
from cibuildwheel.
Hey thanks for this!
Yeah it's a little confusing, but I think that pybind11 is needed for the build. Now, that is correctly listed as a dependency in your setup.py install_requires
, but pip isn't preinstalling that before trying to build the wheel. 🤔
I note that there's a setup.py field called setup_requires
- maybe try adding the pybind11 library to that?
btw, the reason that doing pip install pybind11
before cibuildwheel doesn't work is that they are installed into different python environments. The default python environment on Travis is a homebrew-supplied one I think, that lives at /usr/local/bin/python
. Cibuildwheel uses MacPython distributions which are installed into /Library/Frameworks/Python.framework
from cibuildwheel.
@joerick thanks for the information! Adding setup_requires gets me a different error which I think comes later but is related. Before, After.
Before:
File "/private/var/folders/my/m6ynh3bn6tq06h7xr3js0z7r0000gn/T/pip-vBbxQd-build/setup.py", line 18, in __str__
import pybind11
ImportError: No module named pybind11
----------------------------------------
Failed building wheel for pcst-fast
After
gcc -pthread -fno-strict-aliasing -Wformat -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -I/root/.local/include/python2.6 -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -c src/pcst_fast_pybind.cc -o build/temp.linux-x86_64-2.6/src/pcst_fast_pybind.o -DVERSION_INFO="1.0.2" -std=c++11 -fvisibility=hidden
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
src/pcst_fast_pybind.cc:6:31: fatal error: pybind11/pybind11.h: No such file or directory
#include <pybind11/pybind11.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pcst-fast
Any ideas?
from cibuildwheel.
Could you post the whole 'after' output? I think gcc isn't being invoked with the right include paths... maybe pybind needs to be installed globally for that to work
from cibuildwheel.
Hi @joerick, here's the whole output:
The command "pip install cibuildwheel==0.1.3 pybind11==2.1.0" exited with 0.
48.13s$ cibuildwheel --output-dir wheelhouse
Unable to find image 'quay.io/pypa/manylinux1_x86_64:latest' locally
latest: Pulling from pypa/manylinux1_x86_64
Status: Downloaded newer image for quay.io/pypa/manylinux1_x86_64:latest
+ cd /project
+ for PYBIN in '/opt/python/*/bin'
+ /opt/python/cp26-cp26m/bin/pip wheel . -w /tmp/linux_wheels
DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6
Processing /project
Collecting pybind11==2.1.0 (from pcst-fast==1.0.2)
/opt/_internal/cpython-2.6.9-ucs2/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/opt/_internal/cpython-2.6.9-ucs2/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading pybind11-2.1.0-py2.py3-none-any.whl (109kB)
Saved /tmp/linux_wheels/pybind11-2.1.0-py2.py3-none-any.whl
Skipping pybind11, due to already being wheel.
Building wheels for collected packages: pcst-fast
Running setup.py bdist_wheel for pcst-fast: started
Running setup.py bdist_wheel for pcst-fast: finished with status 'error'
Complete output from command /opt/_internal/cpython-2.6.9-ucs2/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-9aHlBZ-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpaFXl8bpip-wheel-:
running bdist_wheel
running build
running build_py
package init file 'src/__init__.py' not found (or not a regular file)
creating build
creating build/lib.linux-x86_64-2.6
creating build/lib.linux-x86_64-2.6/pcst_fast
copying src/test_pcst_fast.py -> build/lib.linux-x86_64-2.6/pcst_fast
running build_ext
creating tmp
gcc -pthread -fno-strict-aliasing -Wformat -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -c /tmp/tmplasQ5j.cpp -o tmp/tmplasQ5j.o -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
gcc -pthread -fno-strict-aliasing -Wformat -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -c /tmp/tmptKJK0N.cpp -o tmp/tmptKJK0N.o -fvisibility=hidden
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
building 'pcst_fast' extension
creating build/temp.linux-x86_64-2.6
creating build/temp.linux-x86_64-2.6/src
gcc -pthread -fno-strict-aliasing -Wformat -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -I/root/.local/include/python2.6 -I/opt/_internal/cpython-2.6.9-ucs2/include/python2.6 -c src/pcst_fast_pybind.cc -o build/temp.linux-x86_64-2.6/src/pcst_fast_pybind.o -DVERSION_INFO="1.0.2" -std=c++11 -fvisibility=hidden
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
src/pcst_fast_pybind.cc:6:31: fatal error: pybind11/pybind11.h: No such file or directory
#include <pybind11/pybind11.h>
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pcst-fast
It seems to get one step further now. Before "import pybind11" failed. Now that's succeeding, but when it goes to creating build/temp.linux-x86_64-2.6/src
it can't find pybind11.h
which is called for at the top of my c++ file.
How would you install globally?
from cibuildwheel.
@joerick This is how the include is done in the demo version of setup.py provided with pybind11 (adapted slightly). Seems reasonable to me, but I'm not totally sure in what order all of these events occur. Kind of strange that I can build the wheel locally no problem...
class get_pybind_include(object):
"""Helper class to determine the pybind11 include path
The purpose of this class is to postpone importing pybind11
until it is actually installed, so that the ``get_include()``
method can be invoked. """
def __init__(self, user=False):
self.user = user
def __str__(self):
import pybind11
return pybind11.get_include(self.user)
ext_modules = [
Extension(
'pcst_fast',
sources=['src/pcst_fast_pybind.cc', 'src/pcst_fast.cc'],
include_dirs=[
# Path to pybind11 headers
get_pybind_include(),
get_pybind_include(user=True)
],
language='c++'
),
]
Interestingly, someone here seems to be having a similar issue: pybind/python_example#18
from cibuildwheel.
I think that when the project is listed in setup_requires
it's installed into an egg by easy_install, so when your setup script asks pip where to find headers it returns the wrong path. Maybe.
I'm going to look at adding an option CIBW_BEFORE_BUILD
. I'm thinking that if you put CIBW_BEFORE_BUILD=pip install pybind11
that'll solve your problem. We'll see!
from cibuildwheel.
Hi @joerick
Thanks for getting back to me! That sounds like a great option to add for this use case. Keep me posted on how it goes!
from cibuildwheel.
Hey. I've added that feature, it's available on a feature branch of this repo for now. You can install with pip install git+https://github.com/joerick/cibuildwheel@2c0d6e3db1590a9f6902c6d46a8ff27aeb4f1add
.
Would be interested if this allows you to solve your problem!
from cibuildwheel.
@joerick One of two has run, and it didn't work strangely. Here's my travis file by the way. I'm still waiting for the second build. (oh, and here's my setup.py copied from here.
from cibuildwheel.
from cibuildwheel.
Hooray! It worked! Got a new error though, which seems to have to do with the compiler? We don't get those error messages when we compile locally...
The second build errored in a really interesting way because it seems like it was able to build the wheel but not install it. Strange...
Thanks for the help @joerick. What will you do with this new feature? Will you merge into master?
from cibuildwheel.
from cibuildwheel.
@joerick from this log it seems like the flag you added only works for some versions of python?
from cibuildwheel.
Ah yeah, on Mac you need to use pip3
instead of pip
so that the package installs into the right environment.
I added {pip}
for this purpose. If you change your option to {pip} install ...
as the option that should change to pip or pip3 as required.
from cibuildwheel.
Btw, I wrote docs for the option here 2c0d6e3
from cibuildwheel.
@joerick Amazing! One of the travis builds succeeded! I'm not sure I understand how the two of them are split up? The other one seems to fail because it has the wrong compiler? Link to travis page.
Oh I see, it succeeded on mac but failed on linux.
from cibuildwheel.
Looks like it's failing on Python 2.6 because of missing PyCapsule. I think this API isn't available in Python 2.6....
Looks like you need the CIBW_SKIP option too! #3 Alternatively you could use the thunk posted above
from cibuildwheel.
The thunk?
from cibuildwheel.
http://py3c.readthedocs.io/en/latest/capsulethunk.html
from cibuildwheel.
@joerick So I would either need to include that .h file in my project or CIBW_SKIP 2.6? I'd much prefer the latter, but it sounds like CIBW_SKIP will be fairly tough to implement?
from cibuildwheel.
@joerick Looks like I'm building all the wheels now in linux with CIBW_SKIP
- (thanks by the way!) but it seems like it's trying to repair the pybind11 wheel at the end, which is failing: see the log. Is there some way to skip that as well? Perhaps this might be a problem arising from CIBW_BEFORE_BUILD
?
from cibuildwheel.
I think this is because pip wheel
is outputting your dependency wheel as well as the one you want. I've added a commit to restrict auditwheel
and delocate
to the package we're building.
pip install git+https://github.com/joerick/cibuildwheel@d5fa5475d4e8c58000d056f44a9968f68337cf95
p.s. thank you for your patience and help with this! we're ironing out a lot of kinks in this project that will hopefully save a lot of debugging time for future users! 👍
from cibuildwheel.
Looks like your build failed again
https://travis-ci.org/fraenkel-lab/pcst_fast/jobs/221348812
I think I'm guessing the wheel name wrong 🤔
from cibuildwheel.
@joerick you're one step ahead of me.
It's worth noting that when you added CIBW_SKIP
the mac builds started failing (whereas they had been working when you added CIBW_BEFORE...
. I'm not entirely sure why. Check it out.
from cibuildwheel.
🙌 do you mind if I add your repo to the readme as an example?
from cibuildwheel.
Of course! No Problem! I think that particular repo of ours will be around for a long time.
from cibuildwheel.
@joerick interestingly, we're having a bug with python 3.5 / macOS (which is resolved in the python 3.6 wheel):
>>> import pcst_fast
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pcst_fast.cpython-35m-darwin.so, 2): Symbol not found: __PyThreadState_UncheckedGet
Referenced from: /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pcst_fast.cpython-35m-darwin.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pcst_fast.cpython-35m-darwin.so
Seems like the relevant part is Symbol not found: __PyThreadState_UncheckedGet
. Do you know anything about this Symbol? I'm not quite sure where to look for this bug...
from cibuildwheel.
Hmm, never seen that before. Anything suspicious in the build log for that wheel?
from cibuildwheel.
Not that I can tell. That's super weird... How is it possible that this successfully installs the wheel on travis but not on real Macs with python 3.5? Could this have to do with minor versions? (I think it fails on 3.5.1, the wheel was built for 3.5.3...) @joerick
from cibuildwheel.
Related Issues (20)
- Sporadic timeouts in oci_container.py HOT 1
- Troubleshooting Python 3.12 uWSGI wheel HOT 16
- Note about `actions/upload-artifact@v4` HOT 25
- Feature Request: flag to skip build if there is not a numpy wheel available HOT 2
- Access environment variables in config-settings
- No build identifiers selected, even though build_config is cp312-* HOT 2
- [FR] Make it easier to run `cibuildwheel` against unlisted Python builds HOT 7
- Cross compilation yields x86-64 wheel HOT 1
- Failure on ubuntu22.04: sizeof(Big Struct) differs between binary wheel and local "pip install" HOT 1
- Ubuntu: ABI difference between pip install (SYSV) and cibuildwheel (GNU/Linux)? HOT 4
- How to cross-compile dylib on macos? HOT 4
- Multi-line test commands can spuriously pass HOT 4
- Update PyPy to 7.3.15 HOT 7
- GitHub Actions Windows: Invalid --only='""' (fixed in 2.16.5) HOT 6
- Transient failures for Windows AMD64 builds when searching for `pyproject.toml` HOT 10
- Support arm64 macOS runner for GitHub Actions HOT 14
- Uninformative error message - Issue with Windows Build HOT 3
- Allow copying artifacts to `/output` in case of test failure on docker
- Conan package builds HOT 3
- Testing universal2 wheel on arm64 macOS cp38 runner result in `not a supported wheel on this platform` HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cibuildwheel.