Comments (8)
HI, just curious if there has been any progress getting pandas running on 3.13. I built the JIT enabled version of python 3.13 on windows and went to try it with a number of internal tools but ran into build problems trying to get pandas package installed on 3.13. It seems to fail during the numpy portion of the build. Here is the console output from pip. Apologies if this does not belong here or I have made some other major oversight.
Collecting pandas
Using cached pandas-2.2.2.tar.gz (4.4 MB)
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [123 lines of output]
Collecting meson-python==0.13.1
Using cached meson_python-0.13.1-py3-none-any.whl.metadata (4.1 kB)
Collecting meson==1.2.1
Using cached meson-1.2.1-py3-none-any.whl.metadata (1.7 kB)
Collecting wheel
Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB)
Collecting Cython==3.0.5
Using cached Cython-3.0.5-py2.py3-none-any.whl.metadata (3.2 kB)
Collecting numpy>=2.0.0rc1
Using cached numpy-2.0.0rc2.tar.gz (18.3 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
Preparing metadata (pyproject.toml) did not run successfully.
exit code: 1
[88 lines of output]
+ C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\vendored-meson\meson\meson.py setup C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12 C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v\meson-python-native-file.ini
The Meson build system
Version: 1.2.99
Source dir: C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12
Build dir: C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v
Build type: native build
Project name: NumPy
Project version: 2.0.0rc2
C compiler for the host machine: clang (clang 18.1.6 "clang version 18.1.6")
C linker for the host machine: clang link 14.40.33811.0
C++ compiler for the host machine: clang++ (clang 18.1.6 "clang version 18.1.6")
C++ linker for the host machine: clang++ link 14.40.33811.0
Cython compiler for the host machine: cython (cython 3.0.10)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe)
Run-time dependency python found: YES 3.13
Has header "Python.h" with dependency python: YES
Compiler for C supports arguments -fno-strict-aliasing: YES
Compiler for C supports arguments -ftrapping-math: YES
Test features "SSE SSE2 SSE3" : Supported
Test features "SSSE3" : Supported
Test features "SSE41" : Supported
Test features "POPCNT" : Supported
Test features "SSE42" : Supported
Test features "AVX" : Supported
Test features "F16C" : Supported
Test features "FMA3" : Supported
Test features "AVX2" : Supported
Test features "AVX512F" : Supported
Test features "AVX512CD" : Supported
Test features "AVX512_KNL" : Supported
Test features "AVX512_KNM" : Unsupported due to Arguments "-msse, -msse2, -msse3, -mssse3, -msse4.1, -mpopcnt, -msse4.2, -mavx, -mf16c, -mfma, -mavx2, -mno-mmx, -mavx512f, -mavx512cd, -mavx512er, -mavx512pf, -mavx5124fmaps, -mavx5124vnniw, -mavx512vpopcntdq" are not supported
Test features "AVX512_SKX" : Supported
Test features "AVX512_CLX" : Supported
Test features "AVX512_CNL" : Supported
Test features "AVX512_ICL" : Supported
Test features "AVX512_SPR" : Unsupported due to Compiler fails against the test code of "AVX512_SPR"
Configuring npy_cpu_dispatch_config.h using configuration
Message:
CPU Optimization Options
baseline:
Requested : min
Enabled : SSE SSE2 SSE3
dispatch:
Requested : max -xop -fma4
Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
Library m found: NO
Fetching value of define "__MINGW32__" :
Did not find pkg-config by name 'pkg-config'
Found Pkg-config: NO
Run-time dependency scipy-openblas found: NO (tried pkgconfig)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Did not find CMake 'cmake'
Found CMake: NO
Run-time dependency openblas found: NO (tried pkgconfig, pkgconfig, pkgconfig, system and cmake)
Run-time dependency flexiblas found: NO (tried pkgconfig and cmake)
Run-time dependency blis found: NO (tried pkgconfig and cmake)
Run-time dependency blas found: NO (tried pkgconfig and system)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency openblas found: NO (tried pkgconfig, pkgconfig, pkgconfig, system and cmake)
Run-time dependency flexiblas found: NO (tried pkgconfig and cmake)
Run-time dependency lapack found: NO (tried pkgconfig and system)
Program _build_utils/process_src_template.py found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\numpy\_build_utils/process_src_template.py)
Program _build_utils/tempita.py found: YES (C:\Users\jared\OneDrive\Ayrscott\Projects\cpython\PCbuild\amd64\python.exe C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\numpy\_build_utils/tempita.py)
Configuring __config__.py using configuration
..\numpy\meson.build:389: WARNING: The variable(s) 'BLAS_INCLUDEDIR', 'BLAS_LIBDIR', 'BLAS_OPENBLAS_CONFIG', 'BLAS_PCFILEDIR', 'BLAS_TYPE_NAME', 'BLAS_VERSION' in the input file 'numpy\__config__.py.in' are not present in the given configuration data.
Checking for size of "short" : 2
Checking for size of "int" : 4
Checking for size of "long" : 4
Checking for size of "long long" : 8
Checking for size of "float" : 4
Checking for size of "double" : 8
Checking for size of "long double" : 8
Checking for size of "size_t" : 8
Checking for size of "size_t" : 8 (cached)
Checking for size of "wchar_t" : 2
Checking for size of "off_t" : 4
Checking for size of "Py_intptr_t" with dependency python: -1
Checking for size of "PY_LONG_LONG" with dependency python: -1
Has header "complex.h" : YES
Checking for type "complex float" : NO
..\numpy\_core\meson.build:171:4: ERROR: Problem encountered: "complex.h" header does not include complex type complex float
A full log can be found at C:\Users\jared\AppData\Local\Temp\pip-install-__8utyq8\numpy_4569bf49982840bc8c08474080c66a12\.mesonpy-y97ho32v\meson-logs\meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
Encountered error while generating package metadata.
See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
from pandas.
I can't help you with the compiling numpy part, but I just wanted to point out that you can install numpy nightlies, which should have python 3.13 wheels available.
pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
from pandas.
I can't help you with the compiling numpy part, but I just wanted to point out that you can install numpy nightlies, which should have python 3.13 wheels available.
pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy
Hey, thanks for taking the time to reply.
My memory is a little hazy on this now as it was a few weeks back but I believe I tried that approach and was able to get standalone numpy to build using a cherrypicked version but when I went on to and tried to install pandas and requested that it not resolve dependencies (so to use my just built version of numpy) I recall that it seemed to ignore that option and still tried to build whatever specific version of numpy it wanted. It's certainly possible I fudged a CLI option or am misremembering something.
from pandas.
Hey folks!
I'm planning to spend time on fixing any issues related to the new 3.13 free threaded build of CPython in the upcoming weeks. I've already started testing pandas
with it. There's quite a few test failures, but most of them are related to 3.13 and not the free-threaded build itself:
- There's a few failures related to missing PyQT and PyArrow, since both of those are not yet ready for the free threaded build (I didn't try to install Qt and build PyQT from source, that may be worth trying out?).
- There's quite a few failures related to the
csv
module raising aValueError
in a call toregister_dialect
. - The rest of the failures are all related to error messages having changed and thus
pytest.raises
failing.
(This is all tested with the nightly NumPy and Cython wheels)
I'll start investigating further issues that might be there and let everyone know about my progress. I'd also love to talk to anyone who might be trying this out as well, so that we can collaborate better on this effort.
One remaining question: Should I open a new tracking issue for issues specifically related to free threading or should I use this one?
from pandas.
Hi @lysnikolaou,
Thanks for working on this.
I am currently working on fixing regular Python 3.13 test failures in #58776. I'm down to 9 failures there.
I haven't taken a look at free-threading yet, but this is definitely something that's interesting to me.
(thanks for starting on this!)
- There's a few failures related to missing PyQT and PyArrow, since both of those are not yet ready for the free threaded build (I didn't try to install Qt and build PyQT from source, that may be worth trying out?).
Qt is probably not too important here, but if you're able to investigate
(This is all tested with the nightly NumPy and Cython wheels)
I'll start investigating further issues that might be there and let everyone know about my progress. I'd also love to talk to anyone who might be trying this out as well, so that we can collaborate better on this effort.
One remaining question: Should I open a new tracking issue for issues specifically related to free threading or should I use this one?
Please do, it's probably better to separate the general 3.13 compatibility issues from the free threading stuff
P.S.
I wonder how well pandas does with the Python 3.13 JIT 🤔
Is this something you've taken a look at yet?
from pandas.
Hey @lithomas1! Happy to collaborate on this!
I am currently working on fixing regular Python 3.13 test failures in #58776. I'm down to 9 failures there.
Great! The rest of the failures appear to be changes in error messages, so should we expect to get that PR in relatively soon?
Qt is probably not too important here, but if you're able to investigate
Agreed. I'll try to dive into pandas-specific issues first and only later spend time on PyQT if it absolutely necessary. It might make sense for us to skip these tests PyQT related tests when PyQT is not installed.
Please do, it's probably better to separate the general 3.13 compatibility issues from the free threading stuff
I'll open a new tracking issue soon.
I wonder how well pandas does with the Python 3.13 JIT 🤔 Is this something you've taken a look at yet?
No, I haven't tried that out.
from pandas.
Oops, I forgot to finish my sentence
(meant to say if you're able to take at Arrow stuff, that would be nice, as even though Arrow is optional for now, we are pretty tightly integrated with it).
Other than that, a PR adjusting the error messages would be nice.
I have Python 3.13 CI set up in #58776. If you'd like you could target a PR to my fork, or make a new PR with the Github workflow changes there.
from pandas.
meant to say if you're able to take at Arrow stuff, that would be nice, as even though Arrow is optional for now, we are pretty tightly integrated with it
I can check that after we're done with the pandas extension modules themselves, sure.
Other than that, a PR adjusting the error messages would be nice.
I'll get started on that then.
from pandas.
Related Issues (20)
- BUG: `pd.read_excel` gives uninformative error for protected files HOT 2
- BUG: eval fails to process expression when one column name starts with a digit or some special characters HOT 1
- BUG: pandas dataframe column definition or mapping does not cater for upper case values. HOT 1
- BUG: HOT 2
- BUG: Limit param of fillna method does not work for pd.Int64Dtype() HOT 2
- BUG: Failed to import pandas <2.1.0 witn numpy >=2.0.0 HOT 4
- BUG: `Series.clip` does not work with scalar numpy arrays. HOT 5
- BUG: `DataFrame.to_numpy()` unnecessarily upcasts to `object` dtype. HOT 1
- ENH: Python 3.13 free-threading support HOT 4
- BUG: Pandas does not validate some parameters properly when reading CSVs and it causes segmentation faults
- BUG:
- BUG: `DataFrame.eval` fails with TypeError with multiline expr but works when `eval` line by line
- BUG: `DataFrame.sparse.from_spmatrix` hard codes an invalid ``fill_value`` for certain subtypes
- BUG: Dataframe's loc method works incorrectly when selecting a sequence from an index that does not exist. HOT 6
- BUG: index_col in read_csv ignores dtype argument HOT 7
- DOC: clarify how read_csv nrows interacts with header and skiprows argument HOT 3
- BUG: read_csv throws TypeError with iterator, nrows HOT 2
- BUG: [pyarrow] Bizarre overflow error when subtracting two identical `Index` objects. HOT 3
- ENH: Allow to plot weighted KDEs. HOT 3
- BUG: groupby.rank is not supported for large_string[pyarrow] dtype HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.