andersgs / kraken-trawl Goto Github PK

View Code? Open in Web Editor NEW

9.0 9.0 3.0 38 KB

A tool to help create custom `kraken` databases

License: GNU General Public License v3.0

Python 100.00%

kraken-trawl's People

Contributors

Stargazers

Watchers

Forkers

ezozayav pythseq vikash84

kraken-trawl's Issues

Add functionality to control filtering

It should be possible to control what level of assemblies are included. I have thought it would be most useful to have it run in tiers:

all --- just take all assemblies that match to the species/genus list
strict --- only accept refseq genomes
moderate --- if refseq genome(s) is not available, take accept Complete/Chromosome assemblies
liberal --- if neither refseq or Complete/Chromosome assemblies are available, then accept Scaffold/Contig assemblies.

This hierarchical matching means we only keep the best genomes for kraken database. refseq is the gold standard, and anything else could be dodgy (e.g., not be the right species, etc.).

Add some of Kai Blin's ncbi-genome-download functionality

Keep aspera file manifest data as a log

Add automatic injection of Illumina adapters into DB.

Add functionality to download latest taxonomy files when creating a new database.

Remove use of Entrez searches for taxid, and favour searching species ID column in assembly summary file

assembly_summary_genbank.txt has `na` for some assemblies in the `ftp_path` column, which causes the script to fail

Why is there no FTP site for an assembly?

Add functionality to download latest assembly summary genbank file

I've attempted to use kraken-trawl but it hasn't been intuitive for me...I've gotten to this point, but I think I'm typing the wrong command. I downloaded the assemb_file manually with wget ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt, and the file ecoli.txt contains the string Escherichia.

Other info:

$ pip list
biopython (1.70)
click (6.7)
kraken-trawl (0.0.1)
numpy (1.13.1)
pandas (0.20.3)
pip (9.0.1)
python-dateutil (2.6.1)
pytz (2017.2)
setuptools (36.4.0)
six (1.10.0)
wheel (0.29.0)

$ kraken-trawl --assemb_file assembly_summary.txt --taxon_list ecoli.txt --filter_opt liberal
Found ascp
Found the ASPERA key
Found kraken
/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/data/adapter.fasta
kraken-build --add-to-library /Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/data/adapter.fasta --db None
Added Illumina adapters to None
Loading assembly table
/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py:895: DtypeWarning: Columns (20) have mixed types. Specify dtype option on import or set low_memory=False.
  return ctx.invoke(self.callback, **ctx.params)
Total assemblies missing an ftp path was 94497.
Found 6802 hits to Escherichia 
Found 7 hits for Escherichia  in refseq.
For 1 taxa, I found 7 assemblies.
Total assemblies found: 7
Total assemblies expected: 1
Prepping Escherichia coli str. K-12 substr. MG1655 for download.
Prepping Escherichia coli O157:H7 str. Sakai for download.
Prepping Escherichia coli UMN026 for download.
Prepping Escherichia coli IAI39 for download.
Prepping Escherichia coli O83:H1 str. NRG 857C for download.
Prepping Escherichia coli O104:H4 str. 2011C-3493 for download.
Prepping Escherichia vulneris NBRC 102420 for download.
Running the aspera cmd: ascp -d --overwrite=diff -k2 -i /Applications/Aspera\ Connect.app/Contents/Resources/asperaweb_id_dsa.openssh -Q -k1 -T -l500m --host=ftp-private.ncbi.nlm.nih.gov --user=anonftp --mode=recv --file-manifest=text --file-manifest-path=/Users/taylorreiter/projects/LCA_comp/test_human --file-pair-list=aspera_assemblies_src_dest.txt /Users/taylorreiter/projects/LCA_comp/test_human
GCF_000005845.2_ASM584v2_genomic.gbff.gz                                                                                   100% 3244KB (skipped)   --:--    
GCF_000008865.1_ASM886v1_genomic.gbff.gz                                                                                   100% 3735KB (skipped)   --:--    
GCF_000026325.1_ASM2632v1_genomic.gbff.gz                                                                                  100% 3742KB (skipped)   --:--    
GCF_000026345.1_ASM2634v1_genomic.gbff.gz                                                                                  100% 3499KB (skipped)   --:--    
GCF_000183345.1_ASM18334v1_genomic.gbff.gz                                                                                 100% 3210KB (skipped)   --:--    
GCF_000299455.1_ASM29945v1_genomic.gbff.gz                                                                                 100% 3540KB (skipped)   --:--    
GCF_000759795.1_ASM75979v1_genomic.gbff.gz                                                                                 100% 3025KB (skipped)   --:--    
Finisehd downloading all genomes.
Adding Escherichia coli str. K-12 substr. MG1655 to kraken stagging area.
Traceback (most recent call last):
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/bin/kraken-trawl", line 11, in <module>
    load_entry_point('kraken-trawl==0.0.1', 'console_scripts', 'kraken-trawl')()
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/kraken_trawl.py", line 438, in kraken_trawl
    add_isolates(assemb_dic_list, kraken_db)
  File "/Users/taylorreiter/Envs/kraken-trawlEnv/lib/python2.7/site-packages/kraken_trawl/kraken_trawl.py", line 354, in add_isolates
    tmp.id = 'gi|{}'.format(s.annotations['gi'])
KeyError: 'gi'

Use `--file-pair-list` command with aspera

It is possible to feed a list of SRC/DEST to aspera, which will then work on efficiently downloading it all.

The file should look like this:

SRC1
DEST1
SRC2
DEST2
...
SRCN
DESTN

Parallel add genomes to kraken staging area?

It might be worth parallelising the addition of downloaded genomes to the kraken staging area.

Using $HOME/bin versus $HOME/.local/bin

The instructions have --install-option="--install-scripts=$HOME/bin"

Have you told users to add $HOME/bin to their PATH ?

I thought pip defaulted to $HOME/.local/bin ?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.