Comments (6)
@SiddheshBangar - this raises an AssertionError - those shouldn't be exposed to the user (only things like ValueError, TypeError, etc). So this definitely looks like a bug. Also, if you replace None
with "Y"
, then the operation is successful.
from pandas.
At a glance, it seems to me that this should not raise an error. If implementation difficulties make that difficult, then perhaps we can look into other possibilities. Further investigations are welcome!
from pandas.
Hey @nemausus, I just looked into your issue and the problem that i found is that the join
method is trying to align the MultiIndex
levels of the two DataFrames, but the names of the index levels are not aligned correctly.
>>> df1
A
X
3 3 1
>>> df2
B
X
3 3 1
- df1 has a MultiIndex with names ["X", None]
- df2 has a MultiIndex with names [None, "X"]
So it gets confused because the index levels names do not match, even though the structure of the indices is the same.
So if the Data frames have the same names for their multiindex levels this issue can be fixed is what I think, as I tried to test it this is what I found:
>>> df2 = pd.DataFrame({"B": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1 = pd.DataFrame({"A": [1]}, index=pd.MultiIndex.from_tuples([(3, 3)], names=[None, "X"]))
>>> df1.join(df2)
A B
X
3 3 1 1
Let me know if this solves your issue or I can look more further into it, thanks.
from pandas.
Hey @rhshadrach, so you mean to say that the error message should not be prompt or either error message can be fixed somehow and replace it with assertion error message.
from pandas.
Okay, cool thanks @rhshadrach this looks a bit confusing to me at this point but I will try to figure out what the best can I do, if you have any extra information to share feel free to drop down that can help me to resolve this issue, Thanks
from pandas.
It looks like the issue is at
pandas/pandas/core/indexes/base.py
Line 4585 in b162331
# maintain the order of the index levels
if how == "right":
level_order = other_names_list + ldrop_names
else:
level_order = self_names_list + rdrop_names
multi_join_idx = multi_join_idx.reorder_levels(level_order)
Null index names are removed from other_names_list
and self_names_list
for upstream operations and then also as MultiIndex.reorder_levels()
accepts a list of str or int (not None). Therefore, length of level_order
becomes one less than index length.
Note that if the above lines are removed (as per Pandas==2.2.2) then this issue no longer exists.
Perhaps MultiIndex.reorder_levels()
could be extended to accept None type also, and level_order can be updated to include None if existing?
from pandas.
Related Issues (20)
- DOC: Draft PDEPs appear as "Under discussion" in the roadmap
- BUG: Wrong timestamp resolution when parsing timestamp string with comma separated milliseconds HOT 5
- BUILD: HOT 1
- BUG: Series.mul silently returns wrong values with `UInt8` dtype when overflowing the max value range HOT 4
- ENH: Support right hand side operations with Series and custom classes HOT 1
- DOC: New contributor meeting, clarify or remove HOT 5
- BUILD: Pandas 1.5.3 is unusable due to incompatibility with Numpy 2.0.0 HOT 1
- BUG: DatetimeIndex intersection is empty or garbage HOT 4
- BUG: CoW does not seem to work on an index with duplicated labels HOT 10
- BUG: reset_index() looses the frequency of a DatetimeIndex HOT 4
- ENH: pd.concat with keys and ignore_index=True should raise HOT 4
- BUG: HOT 1
- BUG: Error on to_datetime() after running multiple times in jupyter notebook HOT 1
- PERF: Difference in using zipped pickle files HOT 4
- BUG: Error on query function when the column name has # symbol HOT 1
- BUG: Couldn't run sql: 'Connection' object has no attribute 'cursor' HOT 4
- Surprising behavior: set_index cannot set a MultiIndex from a tuple, only a list HOT 4
- BUG: pandas.to_datetime reports incorrect index when failing. HOT 7
- BUG: Error message in read_csv misleading when using decimal="," HOT 1
- DOC: Add Bodo to out-of-core projects in ecosystem HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas.