Comments (3)
bnavigator
commented on July 22, 2024
Same for the development version:
pip install 'numpy>=2.0.0rc1' hypothesis pytest numexpr
pip install --pre --extra-index https://pypi.anaconda.org/scientific-python-nightly-wheels/simple pandas
python -c 'import pandas; pandas.test(extra_args=["-k", "test_eval and numexpr", "-l", "--tb=long"])'
running: pytest -k test_eval and numexpr -l --tb=long /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /var/tmp/pdtestmain/pdtestenv/lib64/python3.11/site-packages/pandas
configfile: pyproject.toml
plugins: hypothesis-6.100.2
collected 204134 items / 198621 deselected / 62 skipped / 5513 selected
...
============================================================================================= short test summary info ==============================================================================================
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-python-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right0-/] - ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestTypeCasting::test_binop_typecasting[numexpr-pandas-float64-left_right1-/] - ValueError: Expression (np.float64(3.0)) / (df) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-python] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
FAILED pdtestenv/lib64/python3.11/site-packages/pandas/tests/computation/test_eval.py::TestOperations::test_simple_arith_ops[numexpr-pandas] - ValueError: Expression (np.float64(1.0)) / (np.float64(1.0)) has forbidden control characters.
============================================================== 10 failed, 5501 passed, 62 skipped, 198621 deselected, 2 xpassed, 4 warnings in 45.77s ==============================================================
INSTALLED VERSIONS
------------------
commit : b4d3309e05e0afb7ee5bd671c2150d1e6eebbb88
python : 3.11.9.final.0
python-bits : 64
OS : Linux
OS-release : 6.8.7-1-default
Version : #1 SMP PREEMPT_DYNAMIC Thu Apr 18 07:12:38 UTC 2024 (5c0cf23)
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : de_DE.UTF-8
LOCALE : de_DE.UTF-8
pandas : 3.0.0.dev0+861.gb4d3309e05
numpy : 2.0.0rc1
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 65.5.0
pip : 24.0
Cython : None
pytest : 8.2.0
hypothesis : 6.100.2
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.10.0
odfpy : None
openpyxl : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None
from pandas.
bnavigator
commented on July 22, 2024
Still an issue with pandas-3.0.0.dev0+991.ga3e751c6b4.
The reason is the explicit type repr for all scalar types in numpy 2:
https://numpy.org/devdocs/release/2.0.0-notes.html#representation-of-numpy-scalars-changed
s = '(df) / (np.float64(3.0))', types = {}, context = {'optimization': 'aggressive', 'truediv': False}, sanitize = True
def stringToExpression(s, types, context, sanitize: bool=True):
"""Given a string, convert it to a tree of ExpressionNode's.
"""
# sanitize the string for obvious attack vectors that NumExpr cannot
# parse into its homebrew AST. This is to protect the call to `eval` below.
# We forbid `;`, `:`. `[` and `__`, and attribute access via '.'.
# We cannot ban `.real` or `.imag` however...
# We also cannot ban `.\d*j`, where `\d*` is some digits (or none), e.g. 1.5j, 1.j
if sanitize:
no_whitespace = re.sub(r'\s+', '', s)
skip_quotes = re.sub(r'(\'[^\']*\')', '', no_whitespace)
if _blacklist_re.search(skip_quotes) is not None:
> raise ValueError(f'Expression {s} has forbidden control characters.')
E ValueError: Expression (df) / (np.float64(3.0)) has forbidden control characters.
context = {'optimization': 'aggressive', 'truediv': False}
no_whitespace = '(df)/(np.float64(3.0))'
s = '(df) / (np.float64(3.0))'
sanitize = True
skip_quotes = '(df)/(np.float64(3.0))'
types = {}
pandas_numexpr_numpy2/lib64/python3.11/site-packages/numexpr/necompiler.py:283: ValueErrorfrom pandas.
bnavigator
commented on July 22, 2024
numexpr does not seem to be installed in the Numpy dev CI runs, so these tests are skipped there.
from pandas.
Related Issues (20)
- ENH: Python 3.13 support HOT 8
- BUG: "styler.format.thousands" option doesn't work for integers HOT 4
- BUG: Pandas 2 is broken! HOT 2
- BUG: 2-sided inplace drop loses freq in DatetimeIndex HOT 4
- BUG: read_orc does not use the provided filesystem for all operations HOT 1
- BUG: pd.to_datetime fails to identify actual date format HOT 4
- BUG: eval fails for ExtensionArray HOT 2
- ENH: Randomised row selection with read_csv() HOT 4
- BUG: read_parquet converts all digits strings to int HOT 2
- Make specific pandas dataframe column immuteable / not changeable HOT 4
- BUG: df.drop_duplicates fails if there is only a single row HOT 3
- Potential regression with PR "PERF: Eliminate circular references in accessor attributes (#58733)" HOT 1
- ENH: support parquet's enum type using Categorical when (de)serializing HOT 3
- ENH: generalize `__init__` on a `dict` to `abc.collections.Mapping` and `__getitem__` on a `list` to `abc.collections.Sequence` HOT 10
- ENH: Add a Series method which checks whether a Series is constant HOT 2
- BUG: df.agg with pd.NamedAgg axis=1 unsupported, but errors differently depending on contents of index HOT 1
- BUG: Segmentation Fault when importing Pandas in python 3.10.14 HOT 3
- BUG: df.agg with df with missing values results in IndexError HOT 3
- BUG: Groupby transformation (cumsum) output dtype depends on whether NA is among group labels HOT 9
- DOC: Docstrings missing from .py files in Sphinxext docs folder HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.