Comments (2)
Additionally, I would suggest strongly restricting the rate of requests when a user asks to download the complete database. A warning should also be provided.
For instance, if downloading the entire CAZy database was only possible when the --complete_download
is set, then the output may look like:
$ cazy_webscraper --complete_download
[WARNING] Downloading the complete CAZy database uses a large amount of bandwidth and may cause the CAZy service to deteriorate for other users.
[WARNING] Please consider downloading only the sequences you need, if possible.
[WARNING] Due to the large size of this request, the rate of requests will be limited to <SOME VALUE> per second to help maintain CAZy service for other users.
from cazy_webscraper.
This is no longer relevant for cazy_webscraper V2 because downloading all of CAZy requires one call to CAZy to download the txt file, therefore, default behaviour is to download all of CAZy - especially as this only takes 10-15 minutes.
from cazy_webscraper.
Related Issues (20)
- Failing to retrieve UniProt data HOT 4
- API missing opt to include 'kingdoms' in output
- Update to `sqlalchemy` 2.x HOT 1
- Crashes when retrieving seqs from NCBI
- No PDB acessions matched and Retrieving no protein structure files HOT 3
- Unexpected error message when retrieving AA UniProt sequences HOT 2
- Add subcommands HOT 1
- Increase unit test coverage
- Crashing when retrieving taxs from NCBI HOT 6
- Incomplete read error HOT 3
- Reduced memory demand HOT 1
- Crashing when retrieving taxs from NCBI - perhaps related to #120 HOT 1
- cazy_webscraper - error downloading database HOT 1
- Error during retrieving taxa info from NCBI HOT 1
- Incorrect parsing of NCBI protein version accession HOT 1
- Fix logger inheritance
- Use NCBI Tax IDs
- Bio.Entrez NotXMLError HOT 1
- Fails to retrieve data from UniProt
- Crashes when retrieving NCBI seqs: http.client.IncompleteRead
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cazy_webscraper.