Simple PhysioNet crawler & downloader with aria2 support to accelerate download speed. aria2 can run on remote server and use tools like webui-aria2 to monitor progress.
usage: pndl [-h] -c HEADER [-o] [-a] [-d] [-v] root_url [dir]
Simple PhysioNet crawler & downloader
positional arguments:
root_url URL to root directory of a PhysioNet archive
dir Directory to store downloads
options:
-h, --help show this help message and exit
required arguments:
-c HEADER, --cookie-header HEADER
Cookies header from PhysioNet
optional arguments:
-o, --output-text Output crawl result to text file
-a, --output-aria2 Output crawl result to destination as aria2 input file
-d, --download Download crawled links using aria2 daemon via JSON-RPC
-v, --verbose
- aria2 daemon running if using
--download
option.
poetry install
poetry run pndl