Comments (9)
Generally speaking, because Pip 20.3 is required for the
universal2
wheel, most packages should provide bothx86_64
and one ofuniversal2
/arm64
wheels for now. When Pip 20.3+ is common on macOS, then it might be possible to ship only theuniversal2
wheel.This advice is from Feb 2021 (pypa/cibuildwheel#577), two years later we should be fine with only
universal2
?
It might be worth noting that before 20.3, pip only supported Python < 3.9 - https://pypi.org/project/pip/20.2.4/. So at the very least, when we drop support for Python 3.8 with Pillow 11, the end user will have to be using pip 20.3.
from pillow-wheels.
If it is helpful to have an immediate solution, here are some universal2 builds for Python 3.10 and 3.11 -
universal2.zip
from pillow-wheels.
Thanks @radarhere ! Great to have those.
from pillow-wheels.
I've created multi-build/multibuild#495 to help with this.
Here are universal2 builds for Python 3.8 and 3.9 - universal2.zip
from pillow-wheels.
Thank you for the PR!
Do we need universal2
in addition to x86_64
and arm64
builds?
I'm mindful we have about 60 wheels per release, and expect other architectures will be added at some point, using resources on PyPI (it costs a donated $1.8M per month to run PyPI!) and increasing the (2 hour+) build times here: ~15m each for x86_64
and arm64
, and ~30m for universal2
, multiplied for each Python version.
Looking at the advice at cibuildwheel (we use multibuild rather than cibuildwheel, but it's generic advice):
x86_64
The traditional wheel for Apple, loads on Intel machines, and on Apple Silicon when running Python under Rosetta 2 emulation.
Due to a change in naming, Pip 20.3+ (or an installer using packaging 20.5+) is required to install a binary wheel on macOS Big Sur.
macOS 11 Big Sur is the oldest supported, so this sets a baseline of pip 20.3 for replacing x86_64
. But we require modern pip for installation anyway, and pip 20.3 is over two years old (Nov 2020).
arm64
The native wheel for macOS on Apple Silicon.
Requires Pip 20.3+ (or packaging 20.5+) to install.
Same pip version.
universal2
This wheel contains both architectures, causing it to be up to twice the size (data files do not get doubled, only compiled code). It requires Pip 20.3 (Packaging 20.6+) to load on Intel, and Pip 21.0.1 (Packaging 20.9+) to load on Apple Silicon.
Same pip version needed for replacing x86_64
, slightly newer pip 21.0.1 (Jan 2021) for replacing arm64
.
File sizes for each architecture are about the same, regardless of Python version:
5.8M Pillow-9.4.0-cp38-cp38-macosx_10_10_universal2.whl
3.2M Pillow-9.4.0-cp38-cp38-macosx_10_10_x86_64.whl
2.9M Pillow-9.4.0-cp38-cp38-macosx_11_0_arm64.whl
5.8M Pillow-9.4.0-cp39-cp39-macosx_10_10_universal2.whl
3.2M Pillow-9.4.0-cp39-cp39-macosx_10_10_x86_64.whl
2.9M Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl
5.8M Pillow-9.4.0-cp310-cp310-macosx_10_10_universal2.whl
3.2M Pillow-9.4.0-cp310-cp310-macosx_10_10_x86_64.whl
2.9M Pillow-9.4.0-cp310-cp310-macosx_11_0_arm64.whl
5.8M Pillow-9.4.0-cp311-cp311-macosx_10_10_universal2.whl
3.2M Pillow-9.4.0-cp311-cp311-macosx_10_10_x86_64.whl
2.9M Pillow-9.4.0-cp311-cp311-macosx_11_0_arm64.whl
universal2
(5.8M) is a bit under the combined size of the other two (6.1M).
I don't think 5.8M is too big for a single combined installer?
Note
The dual-architecture
universal2
has a few benefits, but a key benefit to a universal wheel is that a user can bundle these wheels into an application and ship a single binary.However, if you have a large library, then you might prefer to ship the two single-arch wheels instead -
x86_64
andarm64
. In rare cases, you might want to build all three, but in that case, pip will not download the universal wheels, because it prefers the most specific wheel available.
So they suggest there's limited benefit building all three.
Generally speaking, because Pip 20.3 is required for the
universal2
wheel, most packages should provide bothx86_64
and one ofuniversal2
/arm64
wheels for now. When Pip 20.3+ is common on macOS, then it might be possible to ship only theuniversal2
wheel.
This advice is from Feb 2021 (pypa/cibuildwheel#577), two years later we should be fine with only universal2
?
Or if we want to be cautious, how about x86_64
and universal2
now for the next 9.5.0 release (April 2023)?
And because Python 3.7 can only use x86_64
, and we're dropping 3.7 in the following 10.0.0 release (July 2023), we could at the same time switch to universal2
only? That would simplify config too. (We'd still need x86_64
for PyPy.)
from pillow-wheels.
A possible alternative would just be to document how users can create universal2 wheels themselves from our existing x86_64 and arm64 wheels.
$ python3 -m pip download --only-binary=:all: --platform macosx_10_10_x86_64 Pillow
Collecting Pillow
Using cached Pillow-9.4.0-2-cp39-cp39-macosx_10_10_x86_64.whl (3.3 MB)
Saved ./Pillow-9.4.0-2-cp39-cp39-macosx_10_10_x86_64.whl
Successfully downloaded Pillow
$ python3 -m pip download --only-binary=:all: --platform macosx_11_0_arm64 Pillow
Collecting Pillow
Using cached Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl (3.0 MB)
Saved ./Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl
Successfully downloaded Pillow
$ python3 -m pip install delocate
from delocate.fuse import fuse_wheels
fuse_wheels('Pillow-9.4.0-2-cp39-cp39-macosx_10_10_x86_64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_arm64.whl', 'Pillow-9.4.0-cp39-cp39-macosx_11_0_universal2.whl')
I've created python-pillow/Pillow#6912 for this.
from pillow-wheels.
python-pillow/Pillow#6912 was merged, and the documentation can now be seen under the macOS tab at https://pillow.readthedocs.io/en/stable/installation.html#basic-installation
from pillow-wheels.
That's super helpful, and I think will work just as well for our use case. Thanks!
from pillow-wheels.
Given #359 (comment), and we've documented how to create a combined universal2 wheel (python-pillow/Pillow#6912), I don't see a need to also produce universal2 wheels in addition to the single-architecture ones.
from pillow-wheels.
Related Issues (20)
- delocate.delocating.DelocationError: wheel/PIL/.dylibs/libwebp.7.dylib already exists HOT 2
- Do not upload to rackspace HOT 7
- About link HOT 4
- Reasonable way to run locally? HOT 2
- Travis CI failing due to missing features: fribidi harfbuzz raqm HOT 4
- Musllinux wheels HOT 9
- libpng not properly linked to libz in manylinux wheels? HOT 7
- curl: (28) Failed to connect to sourceware.org port 443: Operation timed out HOT 4
- Enable re-running failed jobs HOT 3
- Travis CI not deploying wheels to GitHub Releases HOT 2
- Stripping the Pillow libraries fails makes pyinstaller binary fail at runtime HOT 13
- Python 3.11 wheel for macOS does not work for x86_64 HOT 13
- Dependency Dashboard HOT 1
- Alpine Linux MUSL Arm64 HOT 5
- x64 wheels are broken HOT 4
- Linux builds fail: no rsync HOT 2
- Depends HOT 1
- Pillow wheel 5.1 broken on OS X 10.11.6 due to liblzma.5.dylib HOT 3
- Xcode 6.4 builds deprecated in Travis CI HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pillow-wheels.