[confusion report] `CIBW_ENVIRONMENT` seems to leak into the test stage about cibuildwheel HOT 5 OPEN

So I confirmed that the env vars set by that param are injected into the test stage, including provisioning the deps: https://github.com/pypa/cibuildwheel/blob/77c57bd/cibuildwheel/windows.py#L549.

Using

before-test = [                                                                                                                                        
  "unset PIP_CONSTRAINT",  # https://github.com/pypa/cibuildwheel/issues/1666                                                                          
]

doesn't work on windows. Still trying to find proper syntax for using a windows-specific command...

from cibuildwheel.

Yes, I understand the confusion. Unfortunately the project grew in such a way that tests were added as part of the 'build' phase, as config options like CIBW_BUILD choose which pythons to build-and-test. Additionally there is no way to just test a wheel without building it first, so the confusion never really came up. Maybe there's some clarification we can do to the environment option though.

As for your specific question, it does suggest the need for an additional CIBW_TEST_ENVIRONMENT option. However, I notice that you are setting PIP_CONSTRAINT. That is actually an option that cibuildwheel sets itself to ensure determinism in the build env that build creates. So it might actually not be having an effect. Is your intention to pin your build backend dependencies? We might have to look at creating a feature for that.

from cibuildwheel.

Yep. Your assessment is correct. We used to run the cythonize program before invoking any Python packaging. That way, we were in control of this specific build dep. Now that I moved that into an in-tree build backend, that control is kinda lost and the current standard way of regaining it is setting the PIP_CONSTRAINT env var, since the CLI flags don't reach the ephemeral PEP 517 build env.

As for setting your own var, I think it's fine for your deps, but might be an undesired side effect for the end-users. Especially, when undocumented... I'd suggest scoping it more carefully.

from cibuildwheel.

So I finally came up with a hack to bridge the gap for us — I injected an early PIP_CONSTRAINT= pip install PyYAML command that ends up building a wheel from sdist with a Cython version matching what PyYAML declares. Pip caches that wheel and any further attempts to install it, just pick it up from cache:
https://github.com/aio-libs/yarl/blob/c907c8e/pyproject.toml#L69-L77

I've made an exclusion for Windows since it always picks up wheels as historically, win32 and win64 wheels have been published by PyYAML, unlike *nix:
https://github.com/aio-libs/yarl/blob/c907c8e/pyproject.toml#L87

I hope to drop this workaround as soon as cibuildwheel starts providing an adequate replacement...

from cibuildwheel.

A clever hack! I did add a +1 to the pypa/build features request for an alternative to PIP_CONSTRAINTS, as I'd like to expose an option for build-dep pinning through CIBW_BUILD_FRONTEND. pypa/build#292 (comment)

In the meantime, it remains an open question over the need for more specific build/test env var options.

On the one hand, the reason this is needed is that PIP_CONSTRAINTS affects the inner workings of both build and cibuildwheel, and that's the source of the issue. If build can support a constraint option, we avoid extra complexity and more options here.

On the other hand, practicality beats purity. Comments/opinions welcome.

from cibuildwheel.