Comments (3)
Hey Bubba,
- With regards to checkM filtering, there are a couple of options:
-
Run with
--checkM_method taxonomy_wf
. This will skip pplacer entirely. The only downside is that it's not robust to archaea (uses universal bacterial single copy genes) -
Filter the genomes with checkM on your own using whatever parameters are best for your system, and then just run dRep with the `--skipCheckM`` option. This won't work if you want dRep to pick the best genomes.
-
Run checkM on your own using whatever parameters are best for your system, make a tab_table of your results, and import them to dRep using
--tab_table
. This will make allow dRep to do the filtering and choose the best genomes based on the checkM results for you.
- That seems like a problem with centrifuge, not checkM. It looks like a lot of the genomes have a
_hits.tsv
file, but not areport.tsv
file. I suspect that in cases where there is no_report.tsv
file, the corresponding_hits.tsv
file is empty. Could you confirm that for me? If so, I'll just write a patch into the program that's able to handle empty files.
Thanks!
-Matt
from drep.
Hey Matt,
1. I tried your first suggestion since I'm not too concerned with archaea at the moment...
In a fresh dir, I --checkM_method taxonomy_wf
:
nice dRep dereplicate_wf . -g genomes/* -o -p 16 --checkM_method taxonomy_wf --run_tax --S_algorithm gANI -sa 0.99 --cent_index /home/ubuntu/scripts/github/centrifuge/indices/b+h+v &> drep.log
It ended up terminating at the gANI step:
$ tail cmd_drep_gANI_gt99.log
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/ubuntu/miniconda3/envs/irep/lib/python3.6/site-packages/pandas/io/parsers.py", line 730, in __init__
self._make_engine(self.engine)
File "/home/ubuntu/miniconda3/envs/irep/lib/python3.6/site-packages/pandas/io/parsers.py", line 923, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/home/ubuntu/miniconda3/envs/irep/lib/python3.6/site-packages/pandas/io/parsers.py", line 1390, in __init__
self._reader = _parser.TextReader(src, **kwds)
File "pandas/parser.pyx", line 373, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:4184)
File "pandas/parser.pyx", line 667, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:8449)
FileNotFoundError: File b'/home/ubuntu/db/ncbi_Blautia_20170418/dRep/gANI_gt99/test_mem1/data/gANI_files/GCF_001404535.1_13414_6_21_genomic.fna_vs_GCF_000153905.1_ASM15390v1_genomic.fna.gANI' does not exist
Indeed the file does not exist, but the two genomes gANI is trying to compare do exist. Nothing seems odd about the files (i.e. not empty or named weird). Is there there an issue with the --checkM_method taxonomy_wf
flag? Attached is the logger.log in case that's helpful.
2. Yeah, this is really odd - but that's what's going on. In the previously mentioned run there are 22 hits:
$ l ../data/centrifuge/*fna_hits.tsv | wc -l
22
And 10 reports:
$ l ../data/centrifuge/*fna_report.tsv | wc -l
10
The centrifuge index I'm using wasn't built on this particular AMI, but I don't think that should matter...
Ok, this is going in all sorts of directions. Sorry!
from drep.
Hey Bubba,
- Try the following command:
/usr/local/bin/ANIcalculator -genome1fna /home/ubuntu/db/ncbi_Blautia_20170418/dRep/gANI_gt99/test_mem1/data/prodigal/GCF_001404535.1_13414_6_21_genomic.fna.fna -genome2fna /home/ubuntu/db/ncbi_Blautia_20170418/dRep/gANI_gt99/test_mem1/data/prodigal/GCF_000153905.1_ASM15390v1_genomic.fna.fna -outfile /home/ubuntu/db/ncbi_Blautia_20170418/dRep/gANI_gt99/test_mem1/data/gANI_files/GCF_001404535.1_13414_6_21_genomic.fna_vs_GCF_000153905.1_ASM15390v1_genomic.fna.gANI -outdir /home/ubuntu/db/ncbi_Blautia_20170418/dRep/gANI_gt99/test_mem1/data/gANI_files/GCF_001404535.1_13414_6_21_genomic.fna_vs_GCF_000153905.1_ASM15390v1_genomic.fna.gANITEMP
Any errors? Are the output files made? That was what dRep tried to run to make that file that doesn't exist
- Could you upload three files for me:
1 + 2) For a genome that has both a _hits.tsv and a _report.tsv file, upload both of those
- For a genome that has only a _hits.tsv file, upload it.
Best,
-Matt
from drep.
Related Issues (20)
- Inquiry on Optimal Workflow for Metagenomic Binning and Dereplication HOT 2
- Drep error with fastANI HOT 1
- Drep error with fastANI HOT 2
- Steps to implement checkm2 quality report to dRep HOT 8
- Halt: the pair-wise MASH clustering step HOT 5
- extra_weight_table flag HOT 3
- Issue with ANImf: 0% ANI despite similar nucleotide sequences HOT 11
- Is there any plan to add skani as ANI calculation method? HOT 2
- drep stoped during clustering HOT 9
- drep conda installation dependencies HOT 4
- dRep: Argument list too long HOT 3
- stuck on checkm HOT 1
- Question abunt the output of dRep HOT 1
- file exists but get os.path.isfile(genome), "{0} is not a file".format(genome) error HOT 10
- ValueError: mash isn't working- make sure its installed HOT 2
- checkM overwriting itself? HOT 4
- checkM failed, properly related HMM file HOT 1
- Contrast of text in dendograms HOT 1
- assert os.path.isfile(genome), "{0} is not a file".format(genome) AssertionError: ......./b1.fasta is not a file HOT 2
- Singularity image
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from drep.