Comments (20)
HI, just curious if there has been any progress getting pandas running on 3.13. I built the JIT enabled version of python 3.13 on windows and went to try it with a number of internal tools but ran into build problems trying to get pandas package installed on 3.13. It seems to fail during the numpy portion of the build. Here is the console output from pip. Apologies if this does not belong here or I have made some other major oversight.
Collecting pandas
Using cached pandas-2.2.2.tar.gz (4.4 MB)
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [123 lines of output]
Collecting meson-python==0.13.1
Using cached meson_python-0.13.1-py3-none-any.whl.metadata (4.1 kB)
Collecting meson==1.2.1
Using cached meson-1.2.1-py3-none-any.whl.metadata (1.7 kB)
Collecting wheel
Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Collecting Cython==3.0.5
Using cached Cython-3.0.5-py2.py3-none-any.whl.metadata (3.2 kB)
Collecting numpy>=2.0.0rc1
Using cached numpy-2.0.0rc2.tar.gz (18.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
Preparing metadata (pyproject.toml) did not run successfully.
exit code: 1
[88 lines of output]
+ C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\vendored-meson\meson\meson.py setup C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12 C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v\meson-python-native-file.ini
The Meson build system
Version: 1.2.99
Source dir: C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12
Build dir: C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v
Build type: native build
Project name: NumPy
Project version: 2.0.0rc2
C compiler for the host machine: clang (clang 18.1.6 "clang version 18.1.6")
C linker for the host machine: clang link 14.40.33811.0
C++ compiler for the host machine: clang++ (clang 18.1.6 "clang version 18.1.6")
C++ linker for the host machine: clang++ link 14.40.33811.0
Cython compiler for the host machine: cython (cython 3.0.10)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe)
Run-time dependency python found: YES 3.13
Has header "Python.h" with dependency python: YES
Compiler for C supports arguments -fno-strict-aliasing: YES
Compiler for C supports arguments -ftrapping-math: YES
Test features "SSE SSE2 SSE3" : Supported
Test features "SSSE3" : Supported
Test features "SSE41" : Supported
Test features "POPCNT" : Supported
Test features "SSE42" : Supported
Test features "AVX" : Supported
Test features "F16C" : Supported
Test features "FMA3" : Supported
Test features "AVX2" : Supported
Test features "AVX512F" : Supported
Test features "AVX512CD" : Supported
Test features "AVX512_KNL" : Supported
Test features "AVX512_KNM" : Unsupported due to Arguments "-msse, -msse2, -msse3, -mssse3, -msse4.1, -mpopcnt, -msse4.2, -mavx, -mf16c, -mfma, -mavx2, -mno-mmx, -mavx512f, -mavx512cd, -mavx512er, -mavx512pf, -mavx5124fmaps, -mavx5124vnniw, -mavx512vpopcntdq" are not supported
Test features "AVX512_SKX" : Supported
Test features "AVX512_CLX" : Supported
Test features "AVX512_CNL" : Supported
Test features "AVX512_ICL" : Supported
Test features "AVX512_SPR" : Unsupported due to Compiler fails against the test code of "AVX512_SPR"
Configuring npy_cpu_dispatch_config.h using configuration
Message:
CPU Optimization Options
baseline:
Requested : min
Enabled : SSE SSE2 SSE3
dispatch:
Requested : max -xop -fma4
Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
Library m found: NO
Fetching value of define "__MINGW32__" :
Did not find pkg-config by name 'pkg-config'
Found Pkg-config: NO
Run-time dependency scipy-openblas found: NO (tried pkgconfig)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Did not find CMake 'cmake'
Found CMake: NO
Run-time dependency openblas found: NO (tried pkgconfig, pkgconfig, pkgconfig, system and cmake)
Run-time dependency flexiblas found: NO (tried pkgconfig and cmake)
Run-time dependency blis found: NO (tried pkgconfig and cmake)
Run-time dependency blas found: NO (tried pkgconfig and system)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency openblas found: NO (tried pkgconfig, pkgconfig, pkgconfig, system and cmake)
Run-time dependency flexiblas found: NO (tried pkgconfig and cmake)
Run-time dependency lapack found: NO (tried pkgconfig and system)
Program _build_utils/process_src_template.py found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\numpy\_build_utils/process_src_template.py)
Program _build_utils/tempita.py found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\numpy\_build_utils/tempita.py)
Configuring __config__.py using configuration
..\numpy\meson.build:389: WARNING: The variable(s) 'BLAS_INCLUDEDIR', 'BLAS_LIBDIR', 'BLAS_OPENBLAS_CONFIG', 'BLAS_PCFILEDIR', 'BLAS_TYPE_NAME', 'BLAS_VERSION' in the input file 'numpy\__config__.py.in' are not present in the given configuration data.
Checking for size of "short" : 2
Checking for size of "int" : 4
Checking for size of "long" : 4
Checking for size of "long long" : 8
Checking for size of "float" : 4
Checking for size of "double" : 8
Checking for size of "long double" : 8
Checking for size of "size_t" : 8
Checking for size of "size_t" : 8 (cached)
Checking for size of "wchar_t" : 2
Checking for size of "off_t" : 4
Checking for size of "Py_intptr_t" with dependency python: -1
Checking for size of "PY_LONG_LONG" with dependency python: -1
Has header "complex.h" : YES
Checking for type "complex float" : NO
..\numpy\_core\meson.build:171:4: ERROR: Problem encountered: "complex.h" header does not include complex type complex float
A full log can be found at C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v\meson-logs\meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
Encountered error while generating package metadata.
See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
from pandas.
It would be awesome for users and maintainers of downstream dependencies if wheels could be uploaded to PyPI directly after the first release candidate (currently targeted for Tuesday, 2024-07-30).
from pandas.
3.13 rc1 is released! This means the ABI is stable and wheels can be uploaded to PyPI.
Tracking for cibuildwheel: pypa/cibuildwheel#1949
What would be fastest feasible course to get wheels on PyPI? Would a 2.2.3 release with 3.13 wheels be possible?
from pandas.
What would be fastest feasible course to get wheels on PyPI? Would a 2.2.3 release with 3.13 wheels be possible?
3.13 support will probably land as part of the pandas 3.0 release (probably Fall 2024)
from pandas.
When testing the wheels, about 80 of these types of errors happen, on Python 3.13 only:
> assert isinstance(left, ExtensionArray), "left is not an ExtensionArray"
E AssertionError: left is not an ExtensionArray
Any idea what causes these? See #59401 for the full logs.
from pandas.
It seems similar errors are going on on the main branch (example).
Edit: It seems this was the last passing wheel build, which ran on July 26th. It did include successful Python 3.13 wheel builds.
from pandas.
@mroeschke do you think there's any way to get Python 3.13 wheels on PyPI relatively soon, at least before the stable Python 3.13 release? Since there are many downstream dependencies waiting on Pandas before being able to update workflows, tests and wheels.
from pandas.
I can't help you with the compiling numpy part, but I just wanted to point out that you can install numpy nightlies, which should have python 3.13 wheels available.
pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
from pandas.
I can't help you with the compiling numpy part, but I just wanted to point out that you can install numpy nightlies, which should have python 3.13 wheels available.
pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
Hey, thanks for taking the time to reply.
My memory is a little hazy on this now as it was a few weeks back but I believe I tried that approach and was able to get standalone numpy to build using a cherrypicked version but when I went on to and tried to install pandas and requested that it not resolve dependencies (so to use my just built version of numpy) I recall that it seemed to ignore that option and still tried to build whatever specific version of numpy it wanted. It's certainly possible I fudged a CLI option or am misremembering something.
from pandas.
Hey folks!
I'm planning to spend time on fixing any issues related to the new 3.13 free threaded build of CPython in the upcoming weeks. I've already started testing pandas
with it. There's quite a few test failures, but most of them are related to 3.13 and not the free-threaded build itself:
- There's a few failures related to missing PyQT and PyArrow, since both of those are not yet ready for the free threaded build (I didn't try to install Qt and build PyQT from source, that may be worth trying out?).
- There's quite a few failures related to the
csv
module raising aValueError
in a call toregister_dialect
. - The rest of the failures are all related to error messages having changed and thus
pytest.raises
failing.
(This is all tested with the nightly NumPy and Cython wheels)
I'll start investigating further issues that might be there and let everyone know about my progress. I'd also love to talk to anyone who might be trying this out as well, so that we can collaborate better on this effort.
One remaining question: Should I open a new tracking issue for issues specifically related to free threading or should I use this one?
from pandas.
Hi @lysnikolaou,
Thanks for working on this.
I am currently working on fixing regular Python 3.13 test failures in #58776. I'm down to 9 failures there.
I haven't taken a look at free-threading yet, but this is definitely something that's interesting to me.
(thanks for starting on this!)
- There's a few failures related to missing PyQT and PyArrow, since both of those are not yet ready for the free threaded build (I didn't try to install Qt and build PyQT from source, that may be worth trying out?).
Qt is probably not too important here, but if you're able to investigate
(This is all tested with the nightly NumPy and Cython wheels)
I'll start investigating further issues that might be there and let everyone know about my progress. I'd also love to talk to anyone who might be trying this out as well, so that we can collaborate better on this effort.
One remaining question: Should I open a new tracking issue for issues specifically related to free threading or should I use this one?
Please do, it's probably better to separate the general 3.13 compatibility issues from the free threading stuff
P.S.
I wonder how well pandas does with the Python 3.13 JIT 🤔
Is this something you've taken a look at yet?
from pandas.
Hey @lithomas1! Happy to collaborate on this!
I am currently working on fixing regular Python 3.13 test failures in #58776. I'm down to 9 failures there.
Great! The rest of the failures appear to be changes in error messages, so should we expect to get that PR in relatively soon?
Qt is probably not too important here, but if you're able to investigate
Agreed. I'll try to dive into pandas-specific issues first and only later spend time on PyQT if it absolutely necessary. It might make sense for us to skip these tests PyQT related tests when PyQT is not installed.
Please do, it's probably better to separate the general 3.13 compatibility issues from the free threading stuff
I'll open a new tracking issue soon.
I wonder how well pandas does with the Python 3.13 JIT 🤔 Is this something you've taken a look at yet?
No, I haven't tried that out.
from pandas.
Oops, I forgot to finish my sentence
(meant to say if you're able to take at Arrow stuff, that would be nice, as even though Arrow is optional for now, we are pretty tightly integrated with it).
Other than that, a PR adjusting the error messages would be nice.
I have Python 3.13 CI set up in #58776. If you'd like you could target a PR to my fork, or make a new PR with the Github workflow changes there.
from pandas.
meant to say if you're able to take at Arrow stuff, that would be nice, as even though Arrow is optional for now, we are pretty tightly integrated with it
I can check that after we're done with the pandas extension modules themselves, sure.
Other than that, a PR adjusting the error messages would be nice.
I'll get started on that then.
from pandas.
Thanks a lot for your work on this 🙏!
Sorry to be the bearer of bad news, but it seems like the Python 3.13 wheel build has been failing for a few days, I created a separate issue about this #59372 .
from pandas.
Thanks! If it can't be backported to Pandas 2.x, I would love for some 3.0 pre-releases to include it as well.
cibuildwheel now uses the latest Python 3.13.0rc1
and build Python 3.13 by default (pypa/cibuildwheel#1950). Next cibuildwheel release should be soon.
from pandas.
from pandas.
Opened an issue for the error above:
from pandas.
NumPy 2.1.0rc1 is now released with Python 3.13 support. It could be added as a required dependency when using Python 3.13.
"numpy>=2.1.0rc1; python_version>='3.13'",
from pandas.
We have a nightly wheel with 3.13 support here: https://anaconda.org/scientific-python-nightly-wheels/pandas
from pandas.
Related Issues (20)
- BUG: subtracting datetime series from datetime dataframe, or datetime dataframe from datetime series, raises TypeError or UFuncTypeError HOT 2
- BUG: read_csv's usecols type hint isn't match with list of strings HOT 2
- BUG: OverflowError: value too large to convert to int when manipulating very large dataframes HOT 6
- DOC: Wrong bug number in what's new v3.0.0 HOT 3
- ENH: Checking gaps for time series HOT 2
- ENH: Support non-categorical values for pandas bar plots when x axis is datetime values HOT 2
- ENH: In pandas.testing.assert_frame_equal, support per-column configuration HOT 5
- DOC: read_csv: date_format HOT 2
- ENH: Breakpoint method for dataframes HOT 1
- BUG: inconsistency when `read_csv` reads MultiIndex with empty values HOT 2
- BUG: .str.contains `na` validation HOT 4
- ENH: Option to configure line wrapping for columns HOT 4
- BUG: inconsistent indices in `GroupByRolling` when selecting or not selecting subset of columns HOT 1
- BUG: provide better error message for pd.Timedelta - pd.Series[Timestamp] HOT 4
- BUG: Wrong result of Kurtosis HOT 2
- BUG: StringDtype conversion to bool changes False to True HOT 3
- Development with Visual Studio Tools 2022 not possible? HOT 3
- ERR: consistent error messages for unsupported reduction operations
- ENH: `DataFrame.struct.explode(column, *, separator=".")` method to pull struct subfields into the parent DataFrame HOT 2
- BUG: `to_markdown` raises `ValueError` when values are `np.array`s with more than one element HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.