scvi and other issues? about scib-pipeline HOT 11 CLOSED

Hi @AlinaKurjan, I'm currently looking into this in #45. Due to an update in scvi to scvi-tools the imports have changed. I'll let you know when the issue is resolved

from scib-pipeline.

The pipeline should now be updated to use the newest scvi-tools version. You can update your environments as follows (assuming you have mamba installed, otherwise feel free to replace mamba with conda):

git pull

# make sure to remove the old dependency
conda activate scib-pipeline-R4.0
mamba remove scvi 
conda deactivate  # make sure to deactivate before the next step

bash envs/create_conda_environments.sh -r 4.0

from scib-pipeline.

@AlinaKurjan I'd be happy to have a chat about tendon integration if you like. We are just looking to build and write up some guidelines for tissue integration, which may help... but there are always tissue-specific challenges.

from scib-pipeline.

Hi @AlinaKurjan,

This looks like an issue with saving HVGs to a separate file for a Seurat run rather than anything to do with scvi or scvi-tools. The issue is being caused by the conversion from anndata to SCE via rpy2. You should be getting the same issue if you start a python session in the conda environment you installed and just do the following:

import anndata2ri
import scanpy as sc
import rpy2.robjects as ro

anndata2ri.activate()
adata = sc.read('/path/to/your/data')
ro.globalenv["adata"] = adata

If the test data runs through fully then, replacing your file name read by:
adata = sc.datasets.pbmc68k_reduced()

Should work again. Could you test this?

It seems that your issue is related to an 'uint' data column in your dataset. R doesn't have this data format, so it can't be converted. This is probably in your adata.obs or your adata.var as it's a pandas series.

Could you print the outputs of adata.obs.dtypes and adata.var.dtypes? One of the columns you have should say something like "UInt32". That needs to be removed or converted.

from scib-pipeline.

Hi @LuckyMD, thank you for a quick reply!

1. I did get the scvi.model error (sorry, misspelled as "module" in the original question), it was just self-explanatory so I didn't include the code for it, apologies for confusion. Here:

Traceback (most recent call last):
  File "/home/a/akurjan/obds_conda/envs/scib-pipeline-R4.0/lib/python3.7/site-packages/scib/integration.py", line 260, in scanvi
    from scvi.model import SCANVI
ModuleNotFoundError: No module named 'scvi.model'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/integration/runIntegration.py", line 67, in <module>
    runIntegration(file, out, run, hvg, batch, celltype)
  File "scripts/integration/runIntegration.py", line 22, in runIntegration
    integrated = method(adata, batch, celltype)
  File "/home/a/akurjan/obds_conda/envs/scib-pipeline-R4.0/lib/python3.7/site-packages/scib/integration.py", line 262, in scanvi
    raise OptionalDependencyNotInstalled(e)
scib.exceptions.OptionalDependencyNotInstalled: 
scvi.model is an optional dependency and not installed by default. Please make sure you install it manually.
ERROR conda.cli.main_run:execute(49): `conda run python scripts/integration/runIntegration.py -i [my long path]/devquads/prepare/unscaled/hvg/a
data_pre.h5ad -o [my long path]/devquads/integration/unscaled/hvg/scanvi.h5ad -b sample --method scanvi -v 2000 -c celltype` failed. (See above
 for error)

Any tips for this?

2. You were correct in that, indeed, there were a few .obs and one .var columns in uint64 format. After converting all of them to int64 it seems like it is running! Will update as it goes.

from scib-pipeline.

env.txt - my environment info, if that helps

Seems like converting the formats resolves the second error, however the pipeline still fails due to the scvi.model not being installed (see above). Could you please advise on the best course of action in this case? Does it mean that the scvi package installation is incomplete?

Thank you for your help!

from scib-pipeline.

On another note... Please forgive that I googled you briefly, @AlinaKurjan... but I see you're working on tendon. Are you already building an integrated tendon atlas? or is your use of scib more for sample-level integration? It would be super cool to see an integrated reference atlas also for more difficult tissues to study, like tendon.

from scib-pipeline.

@LuckyMD As a group, we are indeed exploring our options for building an integrated atlas that would span different tendon types, ages, and health states. I am personally working with human developing tendons, which adds a whole other layer of complexity, and unfortunately the more I try to figure out the best way to process and integrate my data, the more questions I am left with. That is also why I think the scib tool would be invaluable in helping us pick the best approach 🙂 I'd be really keen to get a few of your thoughts on our work later on if you would be interested!

For now, I will attempt to run the pipeline without scvi and scanvi and then compare with those two methods applied manually in a separate environment.

Thank you again for all the help so far.

from scib-pipeline.

@LuckyMD @mumichae Thank you for your help so far, I've run the scvi and scanvi now with no issues. One final note is that I did not get any readouts for trajectory with any methods. What could be the reason here?

from scib-pipeline.

Trajectory requires you to manually annotate (using e.g., sc.tl.dpt) a trajectory per batch that you'd like to check for conservation in the integrated object. If you'd like to do this, you can store the trajectory in a designated adata.obs column. I'd need to check the column name again though.

from scib-pipeline.

Thank you! @mumichae mentioned it needs to be saved as .obs["dpt"]. I'll give that a go. The results are really interesting already, looks like scVI and scANVI are most fitting for my data too :)

from scib-pipeline.