Comments (8)
Double slicing in backed mode (taking a view of a view in backed mode) is not allowed and we now throw an error: 2b622f4
Why don't you do adata_backed[0:2, 0:2]
?
from anndata.
We are using boolean slicing to allow for complex filtering, and currently double slicing with non-integer or non-slice selectors is not allowed.
In other words, this throws an error:
obs_selector=np.array([True, False, ...])
vars_selector=np.array(False, True, ...])
adata_backed[obs_selector, vars_selector]
And the error message informs that we should try double slicing.
from anndata.
OK! Right, double-slicing in memory mode works fine and is currently the only (not nice) way to get submatrices from boolean vectors. In backed-mode, it's quite a bit trickier.
In any-case: if you need this, I'll implement the functionality, maybe even tonight. Then no double slicing is necessary anymore.
from anndata.
At the moment, we can work around it and don't see a need for you to urgently implement. I ran into the bug because I was benchmarking to determine optimal ways to use anndata in cellxgene. I think the best path would be for us to ship our "MVP", and then have a chat with you about performance. Backed mode will either be useful, or not, based upon that. Seem reasonable?
from anndata.
Sounds very reasonable! Let's discuss!
Meanwhile, I think the submatrix extraction via slicing should be relatively straightforward to get via np.ix_()
applied to the data matrix and everything else stays as is. As we discussed this already ages ago and you worked quite a bit on the indexing at the time, @flying-sheep, any bandwidth for doing this? It's essentially only making sure that the index normalization produces non-slices and handles pd.Index
objects appropriately.
from anndata.
Progress was definitely made here, but I'm not sure this issue is totally solved. Double "fancy" indexing over multiple axes isn't supported by h5py datasets. This does work with backed anndata sparse matrices (at least on master).
Side note: It might be possible for zarr dense arrays via get_orthogonal_selection
.
from anndata.
This issue has been automatically marked as stale because it has not had recent activity.
Please add a comment if you want to keep the issue open. Thank you for your contributions!
from anndata.
We throw a meaningful error here and if we ever start supporting it, we’ll announce it.
anndata/anndata/tests/test_hdf5_backing.py
Lines 246 to 250 in bd47cf9
from anndata.
Related Issues (20)
- TypeError: Can't implicitly convert non-string objects to strings HOT 4
- Error concatenating scRNA with visium dataset
- memory usage of concat HOT 3
- Using `Pint` for units HOT 9
- UMAP of gene subset of adata HOT 2
- Error using sc.pl.stacked_violin HOT 1
- NotImplementedError with concat_on_disk
- Issue when setting anndata.X to numpy array HOT 3
- Does anndata.raw can be automatically modified after define it?
- Make the call to `_check_uniqueness()` optional in `_init_as_actual()` (and `__init__()`) HOT 1
- `concat_on_disk` outer join
- `concat_on_disk` merge strategies are untested/not implemented
- Potentially flaky benchmark HOT 2
- Reading Anndata from only parts of h5ad file: Hack solution HOT 8
- 2D indexing (no “:”) in `sparse_dataset` is not lazy HOT 7
- Set of categories changes when slicing AnnData, but not when slicing DataFrame HOT 1
- Gpu failing because of `cp.full` with `np.nan`
- Add path parameter to write_zarr method HOT 8
- GPU writing HOT 1
- GPU concatenation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from anndata.