davised / get_assemblies Goto Github PK

Datasets is much faster and more intuitive to use.

https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/

Hi @davised

I am interested in using this package for getting assemblies. However, I was wondering if there is a way to also obtain the metadata for each assembly? By metadata I mean the assembly statistics and some of the information contained in the general report (BioSample, BioProject, Submitter...).

Cheers,
Pablo

Hi @davised

So I tried downloading some genomes classified as "Agrobacterium rhizogenes". However, I see that other genomes are also downloaded, including those of Bacillus, Enterococcus, Leptospira, Salmonella, Staphylococcus.

I mean it would be easy to depurate these genomes from the collection. I assume this error originates in the database, right? Or is it because of the get_assemblies package?

I ran the following command:
cat metadata.tab | get_assemblies assembly_ids - --function genomes -o fna

It seems that all the 96 genomes were downloaded however I got the following error:
`Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2449, in retrfile
self.ftp.cwd(file)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 550 GCA_001367915.1_10493_1: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1572, in ftp_open
fp, retrlen = fw.retrfile(file, type)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2451, in retrfile
raise URLError('ftp error: %r' % reason) from reason
urllib.error.URLError: <urlopen error ftp error: error_perm('550 GCA_001367915.1_10493_1: No such file or directory')>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 121, in dl_gzip
copy_url(pbar, task_id, uri, filename)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 142, in copy_url
with urlopen(uri) as response:
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1583, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1572, in ftp_open
fp, retrlen = fw.retrfile(file, type)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2451, in retrfile
raise URLError('ftp error: %r' % reason) from reason
urllib.error.URLError: <urlopen error ftp error: URLError("ftp error: error_perm('550 GCA_001367915.1_10493_1: No such file or directory')")>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1565, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1586, in connect_ftp
return ftpwrapper(user, passwd, host, port, dirs, timeout,
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2407, in init
self.init()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2419, in init
self.ftp.cwd(_target)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 550 genomes/all/GCA/001/367/915/GCF_001367915.1_10493_1_6: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/bin/get_assemblies", line 8, in
sys.exit(main())
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 1101, in main
download_genomes(args.o, dl_mapping, args.threads)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 1058, in download_genomes
output = future.result()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/_base.py", line 439, in result
return self.__get_result()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/_base.py", line 391, in __get_result
raise self._exception
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 126, in dl_gzip
copy_url(uri, task_id, uri, filename)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/site-packages/get_assemblies/main.py", line 142, in copy_url
with urlopen(uri) as response:
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1583, in ftp_open
raise exc.with_traceback(sys.exc_info()[2])
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1565, in ftp_open
fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 1586, in connect_ftp
return ftpwrapper(user, passwd, host, port, dirs, timeout,
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2407, in init
self.init()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/urllib/request.py", line 2419, in init
self.ftp.cwd(_target)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 625, in cwd
return self.voidcmd(cmd)
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 286, in voidcmd
return self.voidresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/Users/pablo/opt/anaconda3/envs/get-assemblies2/lib/python3.10/ftplib.py", line 254, in getresp
raise error_perm(resp)
urllib.error.URLError: <urlopen error ftp error: error_perm('550 genomes/all/GCA/001/367/915/GCF_001367915.1_10493_1_6: No such file or directory')>
`

Is the error showing at the end important? I am attaching the log file as well.

Cheers,
Pablo
get_assemblies.log