Can't index into anndata object with integer obs_names about anndata HOT 11 CLOSED

Sorry about the late response: yes, your .iloc/.loc analogy is correct. You'll find a similar behavior to anndata (without .iloc/.loc) also in numpy structured arrays. I'll add your analogy to the docs.

Thank you!

And, yes, we should also throw an error for the integer indexing. Leaving this open for now.

from anndata.

Just cause I feel like my current work-around is silly (converting the integer obs_names to strings), I was looking into why this is happening. Am I correct in thinking the assertion I'm triggering was added since indexing into an AnnData object with a numeric is supposed to be positional, like pandas .iloc method, while indexing with a string/ categorical is supposed to behave like .loc?

from anndata.

I added this warning when people try to work with non-string indices: 51be5d8.

from anndata.

Hi @falexwolf, @ivirshup ,
I was trying to use BBKNN and got this error when trying to concatenate different datasets (see here). Could you please clarify how you managed to fix that?

Thank you!

from anndata.

Sorry about this bug! It might be related to the objects having integer indices: we shouldn't have an error thrown, I guess. But previously, this was presumably inconsistent behavior. @flying-sheep, did you add the following line at some point to _normalize_index, which triggers the error:

assert names.dtype != float and names.dtype != int, \
    'Don’t call _normalize_index with non-categorical/string names'

from anndata.

am I missing something? AFAIK datasets still can’t have integer var_names or obs_names. else it’s impossible to distinguish if e.g. adata[0, :] means adata[adata.obs_names == 0, :] or adata[np.arange(adata.shape[0]) == 0, :].

from anndata.

Thank you for the clarification! I think this error was not thrown in scanpy 1.2.2 (for example here), but it does occur now (scanpy 1.3.6). I guess a solution could be to force the var_names to string.

from anndata.

am I missing something? AFAIK datasets still can’t have integer var_names or obs_names.

@flying-sheep we never threw an error when people set non-string indices; so there will be objects out there that have non-string indices; the assert statement that was probably inserted by you was the first to actually throw an error; before, integer indices were interpreted .iloc - style even if there was an integer index... that's the only way I can imagine that the notebook above broke

from anndata.

Well, I think it’s an useful error to throw. We could add a tip on how to resolve it by removing the index…

from anndata.

Yes, it's a useful error, but it shouldn't be thrown in getitem and it broke the notebook! Instead, an error should be thrown when setting a non-string index; but in that case I accounted for it via a warning, knowing that I would otherwise break things that previously worked.

We shouldn't allow people to have objects in an invalid state and then be surprised by throwing an error later on.

It would be better if here, we output a warning saying that one matches .iloc-style and not on the elements of the index. This is what happened before, it's OK in many cases and allowed the pancreas notebook referenced above to run through. There might be many such notebooks out there and it would be nice to renable them.

from anndata.

OK! so throwing an error here was a bugfix itself:

e83f9db...6a6f13e

I assume that _normalize_index is just not called correctly when there’s integer indices. The fix is therefore surely more involved than just removing that line or converting it into a warning:

I’m pretty sure that integer indices on anndata with integer names were already broken before that and I just added a nicer error message.

from anndata.