davised / get_assemblies Goto Github PK
View Code? Open in Web Editor NEWDownload assembly files from NCBI
License: Other
Download assembly files from NCBI
License: Other
Datasets is much faster and more intuitive to use.
https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/
Hi @davised
I am interested in using this package for getting assemblies. However, I was wondering if there is a way to also obtain the metadata for each assembly? By metadata I mean the assembly statistics and some of the information contained in the general report (BioSample, BioProject, Submitter...).
Cheers,
Pablo
Hi @davised
So I tried downloading some genomes classified as "Agrobacterium rhizogenes". However, I see that other genomes are also downloaded, including those of Bacillus, Enterococcus, Leptospira, Salmonella, Staphylococcus.
I mean it would be easy to depurate these genomes from the collection. I assume this error originates in the database, right? Or is it because of the get_assemblies package?
I ran the following command:
cat metadata.tab | get_assemblies assembly_ids - --function genomes -o fna
It seems that all the 96 genomes were downloaded however I got the following error:
`Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2449, in retrfile
self.ftp.cwd(file)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 550 GCA_001367915.1_10493_1: No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1572, in ftp_open
fp, retrlen = fw.retrfile(file, type)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2451, in retrfile
raise URLError('ftp error: %r' % reason) from reason
urllib.error.URLError: <urlopen error ftp error: error_perm('550 GCA_001367915.1_10493_1: No such file or directory')>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 121, in dl_gzip
copy_url(pbar, task_id, uri, filename)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 142, in copy_url
with urlopen(uri) as response:
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1583, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1572, in ftp_open
fp, retrlen = fw.retrfile(file, type)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2451, in retrfile
raise URLError('ftp error: %r' % reason) from reason
urllib.error.URLError: <urlopen error ftp error: URLError("ftp error: error_perm('550 GCA_001367915.1_10493_1: No such file or directory')")>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1565, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1586, in connect_ftp
return ftpwrapper(user, passwd, host, port, dirs, timeout,
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2407, in init
self.init()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2419, in init
self.ftp.cwd(_target)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 550 genomes/all/GCA/001/367/915/GCF_001367915.1_10493_1_6: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/bin/get_assemblies", line 8, in
sys.exit(main())
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 1101, in main
download_genomes(args.o, dl_mapping, args.threads)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 1058, in download_genomes
output = future.result()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 126, in dl_gzip
copy_url(uri, task_id, uri, filename)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 142, in copy_url
with urlopen(uri) as response:
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1583, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1565, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1586, in connect_ftp
return ftpwrapper(user, passwd, host, port, dirs, timeout,
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2407, in init
self.init()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2419, in init
self.ftp.cwd(_target)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
urllib.error.URLError: <urlopen error ftp error: error_perm('550 genomes/all/GCA/001/367/915/GCF_001367915.1_10493_1_6: No such file or directory')>
`
Is the error showing at the end important? I am attaching the log file as well.
Cheers,
Pablo
get_assemblies.log
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.