Comments (6)
@SiddheshBangar - this raises an AssertionError - those shouldn't be exposed to the user (only things like ValueError, TypeError, etc). So this definitely looks like a bug. Also, if you replace None
with "Y"
, then the operation is successful.
from pandas.
At a glance, it seems to me that this should not raise an error. If implementation difficulties make that difficult, then perhaps we can look into other possibilities. Further investigations are welcome!
from pandas.
Hey @nemausus, I just looked into your issue and the problem that i found is that the join
method is trying to align the MultiIndex
levels of the two DataFrames, but the names of the index levels are not aligned correctly.
>>> df1
A
X
3 3 1
>>> df2
B
X
3 3 1
- df1 has a MultiIndex with names ["X", None]
- df2 has a MultiIndex with names [None, "X"]
So it gets confused because the index levels names do not match, even though the structure of the indices is the same.
So if the Data frames have the same names for their multiindex levels this issue can be fixed is what I think, as I tried to test it this is what I found:
>>> df2 = pd.DataFrame({"B": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1 = pd.DataFrame({"A": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1.join(df2)
A B
X
3 3 1 1
Let me know if this solves your issue or I can look more further into it, thanks.
from pandas.
Hey @rhshadrach, so you mean to say that the error message should not be prompt or either error message can be fixed somehow and replace it with assertion error message.
from pandas.
Okay, cool thanks @rhshadrach this looks a bit confusing to me at this point but I will try to figure out what the best can I do, if you have any extra information to share feel free to drop down that can help me to resolve this issue, Thanks
from pandas.
It looks like the issue is at
pandas/pandas/core/indexes/base.py
Line 4585 in b162331
# maintain the order of the index levels
if how == "right":
level_order = other_names_list + ldrop_names
else:
level_order = self_names_list + rdrop_names
multi_join_idx = multi_join_idx.reorder_levels(level_order)
Null index names are removed from other_names_list
and self_names_list
for upstream operations and then also as MultiIndex.reorder_levels()
accepts a list of str or int (not None). Therefore, length of level_order
becomes one less than index length.
Note that if the above lines are removed (as per Pandas==2.2.2) then this issue no longer exists.
Perhaps MultiIndex.reorder_levels()
could be extended to accept None type also, and level_order can be updated to include None if existing?
from pandas.
Related Issues (20)
- BUG: `Series.combine_first` replaces `None` with `NaN` at index not provided by `other` HOT 3
- BUG: in agg function HOT 1
- BUG: Rounding timedelta to 3 decimal places does not work correctly and is unstable. HOT 2
- BUG: pd.read_sql() converts string "NA" to None HOT 1
- API: timestamp resolution inference - default to one unit (if possible) instead of being data-dependent?
- DOC: punctuation of titles in Titanic data HOT 1
- BUG: `unstack` with `sort=False` scrambles data HOT 3
- BUG: pandas.read_parquet fails to read parquet files with "file://" prefix on Windows HOT 2
- BUG: csv date inconsistencies HOT 2
- Potential performance regression with "BUG: DateTimeIndex.is_year_start unexpected behavior when constructed…" HOT 1
- BUG: output of pandas.util.hash_pandas_object() changed in pandas 2.2.x HOT 2
- BUG: docker build on windows fails; need eol=lf in .gitattributes HOT 2
- BUG: None value in MultiIndex can't be found
- BUG: HDFStore doesn't save datetime64[s] right HOT 1
- ENH: Pivot Table HOT 1
- ENH: Pandas Tensor Data Type HOT 1
- BUG: Accessing a row with loc recasts dtype and can cause values to change depending on other columns HOT 2
- BUG: Reading large CSV files with `pyarrow` when values contain newline character. HOT 2
- DOC: Link to SPEC 0 instead of NEP 29 HOT 2
- BUG: `to_parquet(engine="fastparquet")` can't handle DataFrame with NumPy array values HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.