hey, i am try to build python package and use your marketpalce, but i find it can't find pip of python38

here is the log

+ for PY_VER in '"${arrPY_VERSIONS[@]}"'
+ /opt/python/cp38-cp38m/bin/pip install --upgrade --no-cache-dir pip
/entrypoint.sh: line 30: /opt/python/cp38-cp38m/bin/pip: No such file or directory

how can i solve it ?

      with:
        python-versions: 'cp36-cp36m cp37-cp37m cp38-cp38m cp39-cp39m'

Hi,

Thanks for creating this github action and maintaining it - really appreciated!

I am trying to use it to build a C++ extension that builds with CMake. I've hit an issue that I'm not 100% sure how to fix - my package requires the Python header files, but CMake isn't finding them:

  CMake Error at /tmp/pip-build-env-6h7f921u/overlay/lib/python3.6/site-packages/cmake/data/share/cmake-3.24/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
    Could NOT find Python (missing: Python_LIBRARIES Development
    Development.Embed) (found suitable version "3.6.15", minimum required is
    "3.6.15")

It clearly finds the right version of python; are the Development components also installed in the container? Apologies if I'm just doing something wrong (which is most likely!).

Hello,

I am using your project to build setproctitle packages (see e.g. this job)

I would be interested in testing the packages just built, so pretty much using the same images to run a pip install PACKAGE and a pytest or tox, with the tests available in the cloned repos and the package installed. The actions should run in the image, so we can test properly each architecture.

Any idea about how to do it?

Thank you very much!

Hi!

In my use case I want to switch to the build directory which is created by running PRE_BUILD_COMMAND. I cannot do that by specifying PACKAGE_PATH because cd is attempted before the PRE_BUILD_COMMAND.

I tried to cheat by supplying do stuff && cd new_dir or do stuff; cd new_dir as PRE_BUILD_COMMAND, but it is escaped, so that doesn't work. Doing a cd in a script would be of no use either.

Is there a way to achieve what I need?

P.S. I also looked at changing the PIP_WHEEL_ARGS to specify the actual location of my setup.py (because it is an arg for pip wheel) but . is added to the pip wheel command anyway, so it doesn't help me either. If PIP_WHEEL_ARGS would replace all of the standard args, including ., that would help me.

Currently when wheels are build, dependency wheels are build with it, which (I think) is not necessary. Further, this means that auditwheel will try to fix these wheels as well, which can lead to unexpected problems.

Adding --no-deps to pip wheel should fix this.

@pedrocamargo, following up on your previous issue (#1), I noticed that in a later build the action tried to auditwheel PyQt5, which was missing a dependency, which made the action error and exit... Fixing this, should fix your problem.

PyPy folks maintain a manylinux2010 image which adds pypy installations to the pypa 2010 image, which would be handy to include here. Since these only add installations to the pypa 2010 image, the pypy 2010 image could be used instead of the upstream pypa 2010 without anything lost, but I can see why that might not be exactly what you want to do.

EDIT:

I believe all that needs to be added is one more list entry to create_tags.py, but it's complicated slightly by the fact that the image would come from docker.io/pypywheels/manylinux2010-pypy_x86_64 instead of quay.io/pypa/manylinux2010_x86_64, unlike the rest

My script works for a long time and suddenly broke recently. Not sure if it's a docker issue or the library env setup.

The problem is similar to pypa/manylinux#357

/usr/bin/docker run --name e5c351c92e44619324a9fb1b8eb57bf888be8_4b2a21 --label 1e5c35 --workdir /github/workspace --rm -e pythonLocation -e LD_LIBRARY_PATH -e INPUT_PYTHON-VERSIONS -e INPUT_BUILD-REQUIREMENTS -e INPUT_SYSTEM-PACKAGES -e INPUT_PRE-BUILD-COMMAND -e INPUT_PACKAGE-PATH -e INPUT_PIP-WHEEL-ARGS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/viztracer/viztracer":"/github/workspace" 1e5c35:1c92e44619324a9fb1b8eb57bf888be8  "cp36-cp36m cp37-cp37m cp38-cp38 cp39-cp39" "" "" "" "" "-w ./dist --no-deps"
+ PY_VERSIONS='cp36-cp36m cp37-cp37m cp38-cp38 cp39-cp39'
+ BUILD_REQUIREMENTS=
+ SYSTEM_PACKAGES=
+ PRE_BUILD_COMMAND=
+ PACKAGE_PATH=
+ PIP_WHEEL_ARGS='-w ./dist --no-deps'
+ cd /github/workspace/
+ '[' '!' -z '' ']'
+ '[' '!' -z '' ']'
+ arrPY_VERSIONS=(${PY_VERSIONS// / })
+ for PY_VER in '"${arrPY_VERSIONS[@]}"'
+ /opt/python/cp36-cp36m/bin/pip install --upgrade --no-cache-dir pip
Requirement already up-to-date: pip in /opt/_internal/cpython-3.6.12/lib/python3.6/site-packages (20.2.3)
+ '[' '!' -z '' ']'
+ /opt/python/cp36-cp36m/bin/pip wheel . -w ./dist --no-deps
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /github/workspace
Building wheels for collected packages: viztracer
  Building wheel for viztracer (setup.py): started
  Building wheel for viztracer (setup.py): finished with status 'done'
  Created wheel for viztracer: filename=viztracer-0.8.1-cp36-cp36m-linux_x86_64.whl size=714538 sha256=ec8a6197af611f0e464023df7f7d3dafbf290be9477f496ea0239a2e7d2b39c2
  Stored in directory: /tmp/pip-ephem-wheel-cache-f1611rw4/wheels/cb/ab/8f/fd29149f371a05d1292662735ad3569bffe7dbc23f93e70d91
Successfully built viztracer
+ for PY_VER in '"${arrPY_VERSIONS[@]}"'
+ /opt/python/cp37-cp37m/bin/pip install --upgrade --no-cache-dir pip
/opt/_internal/cpython-3.7.9/bin/python3.7: error while loading shared libraries: libcrypt.so.2: cannot open shared object file: No such file or directory

Currently the wheels are placed in the wheelhouse folder, instead of the standard dist.

I think it would be nice to have a input to specify the destination folder.
The default can remain wheelhouse

I'm using your action to build x86 packages, but I'm not sure how to build them for alternate architectures. Your documentation mentions using alternate docker images, but I'm not sure how to create that architecture in Github actions. Do you have any examples of that?

pypa/manylinux#1332 (comment)

Have tried versions 0.6.0, 0.7.0 and 0.7.1 and getting the same error on all, three that it can't find pip for 3.11

+ for PY_VER in '"${arrPY_VERSIONS[@]}"'
+ /opt/python/cp311-cp311/bin/pip install --upgrade --no-cache-dir pip
/entrypoint.sh: line 44: /opt/python/cp311-cp311/bin/pip: No such file or directory

As far as I have understood this GitHub Action allows to produce fixed wheels and push it automatically to public registry. I wonder if we could take the output of this build process and upload it to private registries, for instance, or to produce artifacts that we could use with actions/upload-artifact@v1. Any ideas?

I need a very brand spanking new version of clang to build my package (the current manylinux image doesn't cut it). So I made a manylinux container at https://hub.docker.com/r/ibellbelli/manylinuxclang with the new clang stuff, and I tried to add that to my call of this action, but it didn't work. I'm not sure if it ever can work with a truly custom docker image, but I'd like it to.

My failing example:

name: pypi deployer
on:
  push:
    tags:
      - "v*" # Push events to matching v*, i.e. v1.0, v20.15.10
jobs:
  Linux-build:
    runs-on: ubuntu-latest
    env:
      TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}
      TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}
      CXX: clang++
    steps:
      - uses: actions/checkout@v2
      - name: checkout submodules
        run: git submodule update --init --recursive
      - name: build and upload manylinux wheels
        uses: RalfG/[email protected]/manylinuxclang
        with:
          python-versions: 'cp37-cp37m cp38-cp38m cp39-cp39m'
          build-requirements: 'cython numpy'
          pip-wheel-args: '-w ./dist --no-deps'

but no dice:

Getting action download info
Failed to resolve action download info. Error: Unable to resolve action `RalfG/[email protected]***%2Fmanylinuxclang`, unable to find version `v0.3.4-ibell***/manylinuxclang`
Retrying in 20.194 seconds
Failed to resolve action download info. Error: Unable to resolve action `RalfG/[email protected]***%2Fmanylinuxclang`, unable to find version `v0.3.4-ibell***/manylinuxclang`
Retrying in 17.82 seconds
Error: Unable to resolve action `RalfG/[email protected]***%2Fmanylinuxclang`, unable to find version `v0.3.4-ibell***/manylinuxclang`

I'm trying to pack my project in wheels.
The recipe for the wheel is in a separate repository.

1. How to register the path to the setup.py in workflow ?
2. How to download the project source code and put it next to the setup.py?

Thanks for creating this action! However, at least one requirement can't be installed using pip on Linux, PyQT5.

Ideally, installation on windows is done like this:
sudo apt-get install python3-pyqt5

Would it make sense having a third parameter to the action with a list of commands to run prior to compilation?

Stack trace:

Collecting PyQt5
Downloading https://files.pythonhosted.org/packages/3a/fb/eb51731f2dc7c22d8e1a63ba88fb702727b324c635***83a32f27f73b8116/PyQt5-5.14.1.tar.gz (3.2MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
ERROR: Complete output from command /opt/_internal/cpython-3.7.6/bin/python /opt/_internal/cpython-3.7.6/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmprhikcrkk:
Preparing wheel metadata: finished with status 'error'
ERROR: Traceback (most recent call last):
File "/opt/_internal/cpython-3.7.6/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 64, in prepare_metadata_for_build_wheel
hook = backend.prepare_metadata_for_build_wheel
AttributeError: module 'sipbuild.api' has no attribute 'prepare_metadata_for_build_wheel'

During handling of the above exception, another exception occurred:

https://github.com/common-workflow-language/schema_salad/actions/runs/3938642519/jobs/6737576186#step:9:570

I guess an apd add findutils is needed somewhere?

Is it possible to change an option to upload to test.pypi instead of pypi ?

I tried full_workflow_example.yml on my actions.
https://github.com/morugu/publish-pypi-example/blob/master/.github/workflows/main.yml

however following error occurred.

Successfully built pypi-example
+ for whl in '/github/workspace/wheelhouse/*-linux*.whl'
+ auditwheel repair '/github/workspace/wheelhouse/*-linux*.whl' --plat manylinux2010_x86_64 -w /github/workspace/wheelhouse/
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot access /github/workspace/wheelhouse/*-linux*.whl. No such file
+ echo 'Repairing wheels failed.'
Repairing wheels failed.
+ auditwheel show '/github/workspace/wheelhouse/*-linux*.whl'
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot access /github/workspace/wheelhouse/*-linux*.whl. No such file

I have no idea how to solve it.

Hi, thanks for this great package.
I used pybind11 rather than Cython.
It seems that this package requires Cython. is there a way to work around?

ERROR: Directory '/github/workspace/' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
+ echo 'Building wheels failed.'
Building wheels failed.
+ exit 1

My package fails to link properly, am I doing something incorrectly here?

https://github.com/dralley/createrepo_c/actions/runs/39456233/workflow

https://github.com/dralley/createrepo_c/runs/446531451?check_suite_focus=true#step:6:470

Got a failure with py38 (py36 & py37 are fine)

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.4',
'GLIBC_2.2.5', 'GLIBC_2.14'}

I'm not sure what's happened here, somehow it's ended up linked against a newer/incorrect libc.so ?

Full results here: https://github.com/LordAro/nml/runs/553326716?check_suite_focus=true

What's gone wrong?

Hi,

I am trying to use RalfG/[email protected]_aarch64 image in github-actions to build manylinux wheels for aarch64 along with qemu, but I am facing error standard_init_linux.go:211: exec user process caused "exec format error" while building the image at step RUN chmod +x /entrypoint.sh, as it is building before running qemu.

Please suggest if I need to add anything to build this image. Thanks.

Build report: https://github.com/odidev/github-test/runs/930701151?check_suite_focus=true

Suggestion: As we are already setting entrypoint.sh to the executable mode, I think we can remove this step from the Dockerfile.

Please share your opinion on the same.

Hi everyone, I am trying to upload a project with C extensions to pypi, this is my first time uploading this kind of platform exclusive project, I have encountered some problems and need help.

I want to use github actions to help me automate the process of publishing my project to pypi, I managed to implement the windows version of the upload, but when coming to linux, I tried to use this repo, but there raised an error that I couldn't understand the error message.

My repo link is GoodManWEN/py-fnvhash-c, where simplified version of the project tree structure is as follow:

.
├── fnvhash_c     #  pypi project name
│   ├── bfilt.py
│   └── __init__.py
├── LICENSE
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
└── src
     ├── cityhash.pyx
     ├── cityhash_setup.py
     └──  include
          └── city.h

On windows system, in a full upload operation, I need to do the following commands:

cd src
cython cityhash_setup.py build_ext --inplace
move *.pyd ../fnvhash_c
cd ..
python setup.py bdist_wheel
twine upload dist/*

And then I tried to switch it into a many linux version, I used the following actions command:

    - name: Build manylinux Python wheels
      uses: RalfG/[email protected]_x86_64
      with:
        python-versions: 'cp36-cp36m cp37-cp37m cp38-cp38 cp39-cp39'
        build-requirements: 'cython'

Got the error message:

Building wheels for collected packages: fnvhash-c
  Building wheel for fnvhash-c (setup.py): started
  Building wheel for fnvhash-c (setup.py): finished with status 'done'
  Created wheel for fnvhash-c: filename=fnvhash_c-dev-cp36-cp36m-linux_x86_64.whl size=4647 sha256=fc10cd549f184c78159b4dca9326676111cae7dafc6dc69b60c6b70a1adfc819
  Stored in directory: /tmp/pip-ephem-wheel-cache-yh13j51a/wheels/cb/ab/8f/fd29149f371a05d1292662735ad3569bffe7dbc23f93e70d91
  WARNING: Built wheel for fnvhash-c is invalid: Metadata 1.2 mandates PEP 440 version, but 'dev' is not
Failed to build fnvhash-c
ERROR: Failed to build one or more wheels
+ echo 'Building wheels failed.'
+ exit 1
Building wheels failed.

One thing I didn't quite understand through the docs is that, theoretically building a c++ plugin and building a pypi wheel are two different builds.When using this action repo, I'm not quite sure when the cython build command is supposed to happen.

How should I adjust my project to make it work properly? Thanks.

https://github.com/common-workflow-language/schema_salad/runs/7371667826?check_suite_focus=true#step:7:131

Any suggestions for a workaround? I initially tried modifying my repo's .dockerignore but that is used for building containers, not running the python-wheels-manylinux-build container

I'm sure the title itself is clear. To explain, I need a build wheel for pyaudio for Py3.7 - 32 Bit - windows 10. I'm confused by the name 'manylinux'. Can I make it work for Windows or is it just Linux?

P.S I don't really need to put it on PyPi. Just need to install on my system.

Thanks You.

Hi,

Is there a plan to support newer manylinux images, for example, manylinux_2_24?

I have the following GitHub action:

https://github.com/kliment/Printrun/pull/1082/files#diff-8efddb34c08d40af8b87450ad25f4cea

The find ... auditwheel call fails with: sh: ./auditwheel: No such file or directory

+ find . -type f -iname '*-linux*.whl' -execdir sh -c 'auditwheel repair '\''{}'\'' -w ./ --plat '\''manylinux1_x86_64'\'' || { echo '\''Repairing wheels failed.'\''; auditwheel show '\''{}'\''; exit 1; }' ';'
sh: ./auditwheel: No such file or directory
Repairing wheels failed.

Printrun-2.0.0rc6-cp37-cp37m-linux_x86_64.whl is consistent with the
following platform tag: "manylinux1_x86_64".

The wheel references external versioned symbols in these system-
provided shared libraries: libc.so.6 with versions {'GLIBC_2.2.5'}

The following external shared libraries are required by the wheel:
{
    "libc.so.6": "/lib64/libc-2.5.so",
    "libpthread.so.0": "/lib64/libpthread-2.5.so"
}

I cannot really see why auditsheel is searched in . rather than on $PATH but is it possible that something is wrong on the docker image?

If I want to build manylinux2010 wheel for a project that requires the Rust build toolchain then it seems I am somewhat out of luck. The manylinux2010 build environment has no system packages for the Rust build toolchain and I can't install it myself using the parameters exposed by this Github action.

It would be useful to have an escape hatch for executing an arbitrary executable (maybe a shell script) in the build environment as part of setup.

Running everything as root can break the next actions in the pipeline.

In my case the issue was caused by the fact that for my package, say foobar, the .egg-file created in the src directory was created by this action and had rights only for root; see the output of ls -al

drwxr-xr-x 4 runner docker 4096 Nov 10 10:13 .
drwxr-xr-x 5 runner docker 4096 Nov 10 10:13 ..
drwxr-xr-x 2 root   root   4096 Nov 10 10:13 foobar.egg-info

Later in the workflow, I also build an sdist using python -m build --sdist. When setuptools then tries to use the .egg-file, it cannot have rights to do so.

Importantly, I have several packages and this error only pops out for the one one that has a C++ extension.

If there is hope to fix this issue, I am happy to deliver a small demo package, or a whole repo, which will demonstrate the issue.

One of my packages won't build because of "too recent versioned symbols", however, it won't tell me which one:

https://github.com/dralley/createrepo_c/runs/447946439?check_suite_focus=true#step:7:321

There are some commands to provide that information -- is there any chance that the CI could expose that information somehow, if the repair fails?

pypa/auditwheel#159