Giter Club home page Giter Club logo

aria2p's Introduction

pawamoy

about language music

πŸͺ Hello world

Hi, I'm TimothΓ©e Mazzucotelli. I develop open-source software, primarily Python tools and libraries. I do this full time: I earn my keep through sponsorships from individual users and companies, which is what makes these projects sustainable and gives you a chance to use them.

You can say hello to me on Mastodon, X, LinkedIn, Matrix, Discord, Telegram, Signal, or by email at [email protected] πŸ“« πŸ‘‹πŸΌ

🌱 Features

  • πŸŽ“ learned computer science at a french university
  • 🐍 about 10 years of experience in development, particularly with Python
  • 🎠 really enjoy sharing code and contributing to other projects
  • πŸ‘” working full time on open-source projects (following a sponsorware strategy)

🌲 Projects

πŸ“˜ Documentation πŸͺ› [Developer] tools #️⃣ Shell stuff
mkdocstrings griffe shellman
mkdocstrings-python aria2p shenv/shenv
mkdocstrings-shell πŸ”’ duty shellm-org/core
mkdocstrings-typescript πŸ”’ failprint shellm-org/cover
mkdocs-autorefs git-changelog shellm-org/daemon
mkdocs-coverage copier-uv shellm-org/debug
mkdocs-manpage copier-pdm shellm-org/format
mkdocs-spellcheck dependenpy shellm-org/loop
mkdocs-pygments πŸ”’ devboard πŸ”’ shellm-org/profiler
markdown-exec pypi-insiders πŸ”’ shellm-org/template
markdown-pycon πŸ”’ insiders-project πŸ”’ shellm-org/trace
happy-path πŸ”’

πŸ‹ See also

Have some green lemon:

green lemon

aria2p's People

Contributors

flipee avatar jonnieey avatar mifluder avatar pawamoy avatar sciuridae0603 avatar sgyzetrov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aria2p's Issues

Fix disk related operations

I made a mistake. All disk operations are assuming the files are on the current filesystem. They will not work for remote systems. I'm not sure how to make it possible for remote as well.

I guess the best thing to do right now is to abort disk operation when the remote is not localhost and print a warning.

Secondly, a workaround to run disk operations on remote is to use them in callbacks on the remote server, with the aria2p listen -c /path/to/callbacks.py functionality. Run the aria2p listen command in a background process (&, maybe followed by disown, or with nohup).

Eventually, maybe we could implement a way to communicate between different aria2p instances, to be able to run disk operations on remotes.

Add command to get/set options

I was thinking of something like:

# get global option
aria2p getopt max-concurrent-downloads

# get downloads options
aria2p getopt dir 390c3d203c46ae0a

# set global option
aria2p setopt max-concurrent-downloads=10

# increment/decrement global integer option
aria2p setopt max-concurrent-downloads=+4
# or +=4? maybe support both. Should be easy as this is a positional argument

# set downloads options
aria2p setopt allow-overwrite=true 390c3d203c46ae0a 2089b05ecca3d829

We should be able to combine this with the "select" and/or "filter" flags of #22.

This issue also requires #14 to be implemented.

Ability to execute instructions on events (download completed, etc.)

Not sure how this one would be implemented, or if it would even be useful.

The JSON-RPC interface does not support notifications over HTTP, but the RPC server will send notifications over Web‐Socket.

Currently we can write shell or Python script that use aria2p and hook them to aria2c.

Add commands/options to the CLI

Add commands to the CLI or options to existing commands:

  • move-files
  • copy-files
  • save-session
  • shutdown

Maybe see API class for more.

Start a wiki

To start we could simply add the solutions found in the "question"-labeled issues. Eventually I'd like to put examples of complex callbacks in there, like a callbacks-based way to implement markers (#19)

aria2p top is not valid

$ aria2p top
aria2p: error: argument : invalid choice: 'top' (choose from 'show', 'call', 'add-magnet', 'add-torrent', 'add-metalink', 'pause', 'pause-all', 'resume', 'resume-all', 'remove', 'rm', 'remove-all', 'purge', 'clear', 'autopurge', 'autoclear')

version 0.71

no top comand ?

Setup CI for Windows

I tried with AppVeyor, but:

  1. got failures, preventing aria2c process to terminate, triggering "resources quota exceeded" error (had to send a mail to support to know what it was: incoming traffic > 7GB)
  2. failed to get logs artifacts since build errored
  3. appveyor refused to let me re-build, trigger a new build, or even delete the project after I forced-push...
  4. at this point I gave up. Unlinked my GitHub account and signed out.

If someone is willing to setup continuous integration for Windows, PRs are welcome.

Improve the download queue

Right now the queue is a simple list. Changes are not immediately reflected, one has to call again api.get_downloads() to synchronize the list. Maybe we would implement some self-synchronizing structure.

Add some details in README for less computer-savvy folks

Although the README.md is comprehensive enough, I did run into a problem when first using aria2p as a library due to not understanding the nature of aria2p. So I added some further explanations in README.md.

I tried to follow CONTRIBUTING.md and file a PR, but poetry keeps acting wired in my docker container. Anyway, a simple update in README.md can be done through an issue, I think :-|

My changes in scripts/templates/README.md

Add type annotations

...and mypy checks:

  • in the Makefile (new rule + set it as a dependency to check rule)
  • in CI (as a job in .gitlab-ci.yml)

We'll need to poetry add -D mypy, add the CI job (allowing failure until it passes) then progressively add type hints.

TUI: add an "options" view

It would be great if we could change options directly from within the interface.

The options would be temporary to start, only applied during the time the process is alive. Maybe later we could add an option to "save" them inside of the aria2 config file.

TUI: commands

It could be nice to run commands directly from within the interface, like in vim/tmux/etc.

User would hit : and a prompt would appear, with auto-completion, etc.

The main use case would be to change options, but we can imagine more use-cases.

Add option to top interface to change update interval time

  1. add a parameter update_interval to init method of class interface.Interface. This value is used to change the sleep value: self.sleep = update_interval / self.frames
    • Example with 1 second interval (default): sleep = 1 / 200 = 0.005
    • Example with 2 seconds interval: sleep = 2 / 200 = 0.01
  2. add an option to the top subparser of the main CLI parser (in cli.get_parser function). Something like "-i", "--update-interval", type=float, default=1.0
  3. write tests in tests/test_interface.py
  4. update documentation

Dynamic downloads object

Would it be feasible to make the aria2p.downloads.Download class dynamic?

I.e. being able to return the current status of it's properties, not just the status when the object was first instantiated?

Below is an example of what I mean.

Using aria2p as a module, you can begin a bittorrent download via

import aria2p

# First initializing the API
api = aria2p.API(
            aria2p.Client(
            host="http://localhost",
            port=6800,
            secret=""
            )
# Then adding the magnetlink
download = api.add_magnet('some_magnet_link')

The download object returned here refers to the initial metadata download.
The download object returns 0% progress in perpetuity, even after the meta download has finished and the main download has begun.

>>> download.progress_string()
0.00%
>>> download.is_complete
False

To get the current download status you can:

# Get the GID for the download
>>> meta_gid = download.gid
# Then reinstantiate the download object for a more up-to-date status
>>> download = api.get_download(meta_gid)
>>> download.progress_string()
50%
>>> download = api.get_download(meta_gid)
>>> download.progress_string()
100%

However this is obviously less clean than just having to call the original download object.

Add name/value verification for options (see man page)

Global only options:

  • bt-max-open-files
  • download-result
  • keep-unfinished-download-result
  • log
  • log-level
  • max-concurrent-downloads
  • max-download-result
  • max-overall-download-limit
  • max-overall-upload-limit
  • optimize-concurrent-downloads
  • save-cookies
  • save-session
  • server-stat-of

Gloab/specific options:

  • all-proxy
  • all-proxy-passwd
  • all-proxy-user
  • allow-overwrite
  • allow-piece-length-change
  • always-resume
  • async-dns
  • auto-file-renaming
  • bt-enable-hook-after-hash-check
  • bt-enable-lpd
  • bt-exclude-tracker
  • bt-external-ip
  • bt-force-encryption
  • bt-hash-check-seed
  • bt-load-saved-metadata
  • bt-max-peers
  • bt-metadata-only
  • bt-min-crypto-level
  • bt-prioritize-piece
  • bt-remove-unselected-file
  • bt-request-peer-speed-limit
  • bt-require-crypto
  • bt-save-metadata
  • bt-seed-unverified
  • bt-stop-timeout
  • bt-tracker
  • bt-tracker-connect-timeout
  • bt-tracker-interval
  • bt-tracker-timeout
  • check-integrity
  • conditional-get
  • connect-timeout
  • content-disposition-default-utf8
  • continue
  • dir
  • dry-run
  • enable-http-keep-alive
  • enable-http-pipelining
  • enable-mmap
  • enable-peer-exchange
  • file-allocation
  • follow-metalink
  • follow-torrent
  • force-save
  • ftp-passwd
  • ftp-pasv
  • ftp-proxy
  • ftp-proxy-passwd
  • ftp-proxy-user
  • ftp-reuse-connection
  • ftp-type
  • ftp-user
  • gid
  • hash-check-only
  • header
  • http-accept-gzip
  • http-auth-challenge
  • http-no-cache
  • http-passwd
  • http-proxy
  • http-proxy-passwd
  • http-proxy-user
  • http-user
  • https-proxy
  • https-proxy-passwd
  • https-proxy-user
  • lowest-speed-limit
  • max-connection-per-server
  • max-download-limit
  • max-file-not-found
  • max-mmap-limit
  • max-resume-failure-tries
  • max-tries
  • max-upload-limit
  • metalink-base-uri
  • metalink-enable-unique-protocol
  • metalink-language
  • metalink-location
  • metalink-os
  • metalink-preferred-protocol
  • metalink-version
  • min-split-size
  • no-file-allocation-limit
  • no-netrc
  • no-proxy
  • parameterized-uri
  • pause-metadata
  • piece-length
  • proxy-method
  • realtime-chunk-checksum
  • referer
  • remote-time
  • remove-control-file
  • retry-wait
  • reuse-uri
  • rpc-save-upload-metadata
  • seed-ratio
  • seed-time
  • split
  • ssh-host-key-md
  • stream-piece-selector
  • timeout
  • uri-selector
  • use-head
  • user-agent

Specific only options:

  • checksum
  • index-out
  • out
  • pause
  • select-file

HTOP-like interface

It would be great to provide a HTOP-like interface for showing the current downloads and the progress. Global graph for dl/up speed, collapsable graphs for each dl, sortable columns, customization with config file (columns and their order, colors), support for mouse and keys, etc.

Option to hide metadata downloads

Not sure if this is wanted anymore. Maybe for completed metadata downloads only? Closing for now. Comment if you're interested in this.

Issue with "Max retries exceeded with url: /jsonrpc"

-> already installed aria2c,aria2& aria2p

import aria2p
aria2 = aria2p.API(
aria2p.Client(
host="http://localhost",
port=6800,
secret=""
)
)
m=aria2.add_magnet("magnet:?xt=dghgbh.......")

output
[ERROR] HTTPConnectionPool(host='localhost', port=6800): Max retries exceeded with url: /jsonrpc (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10a2fc2b0>: Failed to establish a new connection: [Errno 61] Connection refused',))

set options download name not work

this code:
aria2.add_uris(uris=[src, ], options={'dir': download_path, 'download': {'name': file_name} })

the download_path is work, but download file name is not work.

define file name

When I download a file, how do I specify the filename instead of using the original filename

OSError: [WinError 123] when use aria2p add <uri> command

This is really wierd. What happened to the URI?

aria2p add 'https://example.com/some/path/filename.ext' .
Traceback (most recent call last):
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\mohan\AppData\Local\Programs\Python\Python36-32\Scripts\aria2p.exe\__main__.py", line 9, in <module>
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\site-packages\aria2p\cli.py", line 108, in main
    return subcommands.get(subcommand)(api, **kwargs)
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\site-packages\aria2p\cli.py", line 403, in subcommand_add
    if path.exists():
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\pathlib.py", line 1334, in exists
    self.stat()
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\pathlib.py", line 1156, in stat
    return self._accessor.stat(self)
  File "c:\users\mohan\appdata\local\programs\python\python36-32\lib\pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: "'https:\\example.com\\some\\path\\filename.ext'"

SVG in README

The SVG in the README made my fan spin really hard and made the page overall irresponsive. Consider replacing it with an asciinema

followed_by not being updated

A metadata download object returned from the API module will not return the followed_by list, even after completion, even with download.live and download.update() methods, for example:

import time

magnetlink = 'magnet:?...'
download = api.add_magnet(magnetlink, options={'dir': '/tmp'})
while not download.live.is_complete:
    time.sleep(5)
print(download.followed_by)

Will always return []
However, the following will return the correct followed_by download objects:

import time

magnetlink = 'magnet:?...'
download = api.add_magnet(magnetlink, options={'dir': '/tmp'})
while not download.live.is_complete:
    time.sleep(5)
download = api.get_download(download.gid)
print(download.followed_by)

Is this expected behaviour? I was under the impression that the Download.live and Download.update() methods would ideally return the updated followed_by list.

Accept multiple positional arguments for add subcommands

If we want to add several magnets from a file, we have to use a shell loop:

while read -r magnet; do
  aria2p add-magnet "$magnet"
done < magnets_file

It would be easier if we were able to do this:

aria2p add-magnet $(cat magnets_file)
# maybe also support one multi-line string:
aria2p add-magnet "$(cat magnets_file)"

Or even provide a flag to load from a file:

aria2p add-magnet -f magnets_file

[BUG] Download.bittorrent is never None

When I try to download something with HTTP (not torrent), I expect the Download.bittorrent to be None however this is never the case.
Here is Download._struct for https://speed.hetzner.de/100MB.bin

{'completedLength': '0', 'connections': '1', 'dir': '/home/shivam/downloads//816/', 'downloadSpeed': '0', 'files': [{'completedLength': '0', 'index': '1', 'length': '0', 'path': '', 'selected': 'true', 'uris': [{'status': 'used', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}, {'status': 'waiting', 'uri': 'https://speed.hetzner.de/100MB.bin'}]}], 'gid': 'be0fa29b8fbc0c96', 'numPieces': '0', 'pieceLength': '1048576', 'status': 'active', 'totalLength': '0', 'uploadLength': '0', 'uploadSpeed': '0'}

While here is the Download._struct when a torrent is added:

{'bittorrent': {'announceList': [['http://nyaa.tracker.wf:7777/announce'], ['udp://open.stealth.si:80/announce'], ['udp://tracker.opentrackr.org:1337/announce'], ['udp://tracker.coppersurfer.tk:6969/announce'], ['udp://exodus.desync.com:6969/announce']]}, 'completedLength': '0', 'connections': '0', 'dir': '/home/shivam/downloads//817/', 'downloadSpeed': '0', 'files': [{'completedLength': '0', 'index': '1', 'length': '0', 'path': '[METADATA][Anime Time] Princess Connect! ReDive - 1 [720p].mp4', 'selected': 'true', 'uris': []}], 'gid': 'df71125cecd63d13', 'infoHash': '985e7f658509dca05b6b1560a4213950b11b7373', 'numPieces': '0', 'numSeeders': '0', 'pieceLength': '16384', 'seeder': 'false', 'status': 'active', 'totalLength': '0', 'uploadLength': '0', 'uploadSpeed': '0'}

bug with secret-token given

aria2c start up, no token daemon ok

$ aria2p call aria2.getVersion
{"enabledFeatures": ["Async DNS", "BitTorrent", "Firefox3 Cookie", "GZip", "HTTPS", "Message Digest", "Metalink", "XML-RPC"], "version": "1.33.1"}
`

aria2c start up, token daemon bug
$aria2p -s token call aria2.getVersion
Unauthorized

Python version requirement inconsistencies

The documentation states 3.6 and above, yet the .toml file states "~3.6" which indicates 3.6.* but not 3.7

Found this out when trying to install the package on my system that is running python 3.7.

Are there any known issues on 3.7 or is it safe to change the required version to "^3.6"?

Replace show with a list command

The list command lists all downloads, sorts them, limits the result, etc.
It can also be used to quickly see the information about specific downloads in the terminal: it accepts gids or filter patterns.

Maybe do this:

  • By default only outputs downloads names.
  • Option -l, --long for additional information (--format=long).
  • Option -f, --format with choices and custom string.
  • Option -s, --sort with choices and custom sort.
  • Option -H, --human-readable.

Abstract away the difference between a download and a download result, i.e. remove and purge

I think it would make aria2p easier to understand and use. Currently we "remove" downloads and "purge" download results. I'd like to get rid of the "purge" methods and only use "remove". The "remove" methods would be smart enough to know if we have to internally purge or remove, based on the download object status.

I think it would simplify things, but maybe it's a not a good idea. Any thoughts, users of aria2p?

library: use timeouts

Sometimes network calls can block forever if no timeouts are specified. For example here

return requests.post(self.server, data=payload).json()

It would be good if a timeout could be passed to requests.

Maybe it could be specified globally for the client like Client(..., timeout=60) and then use requests.post(..., timeout=self.timeout).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.