andersgs / kraken-trawl Goto Github PK
View Code? Open in Web Editor NEWA tool to help create custom `kraken` databases
License: GNU General Public License v3.0
A tool to help create custom `kraken` databases
License: GNU General Public License v3.0
It should be possible to control what level of assemblies are included. I have thought it would be most useful to have it run in tiers:
This hierarchical matching means we only keep the best genomes for kraken
database. refseq
is the gold standard, and anything else could be dodgy (e.g., not be the right species, etc.).
Why is there no FTP site for an assembly?
I've attempted to use kraken-trawl but it hasn't been intuitive for me...I've gotten to this point, but I think I'm typing the wrong command. I downloaded the assemb_file
manually with wget ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt
, and the file ecoli.txt
contains the string Escherichia
.
Other info:
$ pip list
biopython (1.70)
click (6.7)
kraken-trawl (0.0.1)
numpy (1.13.1)
pandas (0.20.3)
pip (9.0.1)
python-dateutil (2.6.1)
pytz (2017.2)
setuptools (36.4.0)
six (1.10.0)
wheel (0.29.0)
$ kraken-trawl --assemb_file assembly_summary.txt --taxon_list ecoli.txt --filter_opt liberal
Found ascp
Found the ASPERA key
Found kraken
/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/data/adapter.fasta
kraken-build --add-to-library /Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/data/adapter.fasta --db None
Added Illumina adapters to None
Loading assembly table
/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py:895: DtypeWarning: Columns (20) have mixed types. Specify dtype option on import or set low_memory=False.
return ctx.invoke(self.callback, **ctx.params)
Total assemblies missing an ftp path was 94497.
Found 6802 hits to Escherichia
Found 7 hits for Escherichia in refseq.
For 1 taxa, I found 7 assemblies.
Total assemblies found: 7
Total assemblies expected: 1
Prepping Escherichia coli str. K-12 substr. MG1655 for download.
Prepping Escherichia coli O157:H7 str. Sakai for download.
Prepping Escherichia coli UMN026 for download.
Prepping Escherichia coli IAI39 for download.
Prepping Escherichia coli O83:H1 str. NRG 857C for download.
Prepping Escherichia coli O104:H4 str. 2011C-3493 for download.
Prepping Escherichia vulneris NBRC 102420 for download.
Running the aspera cmd: ascp -d --overwrite=diff -k2 -i /Applications/Aspera\ Connect.app/Contents/Resources/asperaweb_id_dsa.openssh -Q -k1 -T -l500m --host=ftp-private.ncbi.nlm.nih.gov --user=anonftp --mode=recv --file-manifest=text --file-manifest-path=/Users/taylorreiter/projects/LCA_comp/test_human --file-pair-list=aspera_assemblies_src_dest.txt /Users/taylorreiter/projects/LCA_comp/test_human
GCF_000005845.2_ASM584v2_genomic.gbff.gz 100% 3244KB (skipped) --:--
GCF_000008865.1_ASM886v1_genomic.gbff.gz 100% 3735KB (skipped) --:--
GCF_000026325.1_ASM2632v1_genomic.gbff.gz 100% 3742KB (skipped) --:--
GCF_000026345.1_ASM2634v1_genomic.gbff.gz 100% 3499KB (skipped) --:--
GCF_000183345.1_ASM18334v1_genomic.gbff.gz 100% 3210KB (skipped) --:--
GCF_000299455.1_ASM29945v1_genomic.gbff.gz 100% 3540KB (skipped) --:--
GCF_000759795.1_ASM75979v1_genomic.gbff.gz 100% 3025KB (skipped) --:--
Finisehd downloading all genomes.
Adding Escherichia coli str. K-12 substr. MG1655 to kraken stagging area.
Traceback (most recent call last):
File "/Users/taylorreiter/Envs/kraken-trawlEnv/bin/kraken-trawl", line 11, in <module>
load_entry_point('kraken-trawl==0.0.1', 'console_scripts', 'kraken-trawl')()
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/kraken_trawl.py", line 438, in kraken_trawl
add_isolates(assemb_dic_list, kraken_db)
File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/kraken_trawl.py", line 354, in add_isolates
tmp.id = 'gi|{}'.format(s.annotations['gi'])
KeyError: 'gi'
It is possible to feed a list of SRC/DEST to aspera, which will then work on efficiently downloading it all.
The file should look like this:
SRC1
DEST1
SRC2
DEST2
...
SRCN
DESTN
It might be worth parallelising the addition of downloaded genomes to the kraken
staging area.
The instructions have --install-option="--install-scripts=$HOME/bin"
Have you told users to add $HOME/bin
to their PATH ?
I thought pip defaulted to $HOME/.local/bin
?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.