Giter Club home page Giter Club logo

rtorrent-ps-ch_setup's Introduction

Automated rTorrent-PS-CH configuration

An almost completely automated setup with rTorrent-PS-CH that doesn't need any additional UI only an SSH client (with the help of tmux), with category support, requiring the least amount of human interaction, including config files/scripts/instrucions for FTP, Samba, email reporting and many more.

Extended Canvas Screenshot

"Interesting. Can I read more about it?"

Sure, take a look at the WIKI pages for more information (that are also included in docs directory).

rtorrent-ps-ch_setup's People

Contributors

chaz6 avatar chros73 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rtorrent-ps-ch_setup's Issues

Make a comment on current implementation of d.move_data method

Make a comment on current implementation of d.move_data method (that is used with download.finished and download.hash_done events) regarding to stop and start methods in the code to leave it as it is until it's working fine!!!

Current code is:
method.insert = d.move_data, simple|private, "d.directory.set=$d.custom=data_dir;execute=mv,-f,$d.base_path=,$d.custom=data_dir;d.stop=;d.start="

When it's changed to the more sensible form (moving d.stop= at the beginning) of:
method.insert = d.move_data, simple|private, "d.stop=;d.directory.set=$d.custom=data_dir;execute=mv,-f,$d.base_path=,$d.custom=data_dir;d.start="
it triggers a really nasty bug in tracker controller: there will be stucked http/https requests for newly added torrents, and new requests (announce/scrape) can't be made any more for those torrents, resulting less peers. Restarting the client solves the issue for those torrents.
Interestingly, the other method (d.move_to.helper) that is using this technique doesn't suffer from this (it's used manually not by the above mentioned events) so it's probably regarding to changing the state of torrents.

A good header of trackers should look like this:
Trackers: [Key: xxxxxxxx] [req ]
The bug results header of trackers like this:
Trackers: [Key: xxxxxxxx] [ ]

Add additional color configs

Using pyroscope idea (pyroscope/rtorrent-ps@d10a598), add those color config files to the setup and put out the existing one as well.

Create a new command called import.pyro.rc to easily import additional config files from ~/.pyroscope directory. Usage:

import.pyro.rc="color_scheme8.rc"

Naming

A way better (non-confusing) name for this repo would be something like "rtorrent-ps-cfg" or "rtorrent-ps-setup", btw.

Add support for delete-queue

Add support for .delqueue directory and refactor corresponding code.

The purpose of delete-queue is to be able to specify torrents (in the form of symlinks pointing to their metafile in this .delqueue directory) that will be deleted at first by a future queue script.

  1. Add new boolean flag to store this: d.custom=delqueue
  2. Create the following methods:
  • d.add_to_delqueue adds the selected torrent to delqueue only if it has the unsafe_data flag
  • d.remove_from_delqueue removes the selected torrent from delqueue
  • d.fix_delqueue_flag fixes delqueue flag, apply only if unsafe_data flag and symlink in delqueue dir exists, remove it otherwise .
    Run this once per day: it's useful when a script creates these symlinks and we want to update this property.
  1. Include delqueue support when data deleting happens.
  2. This d.custom=delqueue flag (along with d.custom=unsafe_data flag) will be visible in the patched rtorrent-ps: chros73/rtorrent-ps-ch#3
  3. Modify existing d.move_to manual moving method to support this new flag and reorder its parameters:
  • it receives 4 arguments: dirname,specialgroup,unsafe_data,delqueue
  • d.move_to=rotating,1,1, or d.move_to=rotating,1,1,1
  • d.move_to=fullseed,1,,
  • d.move_to=unsafe,,1, or d.move_to=unsafe,,1,1
  • d.move_to=others,,,

Add support for possibly missing d.base_path attribute

See for more info here: rakshasa/rtorrent#448
This particular bug didn't affect this setup (since it never stops a torrent) but it's always good to be prepared the worst case.
Similarly d.base_filename is empty as well in this case.

Solution:

  • replace d.base_filename with d.name
  • replace d.base_path with a method that will give back the full path of data of a torrent all the time.

Bind key to toggle unsafe_data attribute

Bind } key to toggle unsafe_data attribute (See #25). (This key should not be easily accessible.)

It sets it in the following order: ,1(deletable) ,2 (deletable in delete queue)

Regularly send scrape request to trackers

By default, rtorrent only sends scrape request (that can give back the numbers of peers, seeds, downloads for a specific torrent) to trackers when a torrent is added or the client was restarted. More info here: rakshasa/rtorrent#413
Probably this info only cosmetics but good to see these changing.
Use a custom field (tm_last_scrape) to store the last scrape time to be able to save into session (the inbuilt t.scrape_time_last property is lost upon reboot).
Let's send these requests regularly:

  • try to balance the calls to not fire up them at the same time
  • every 10 minutes for transferring torrents (uploading/downloading)
  • every 12 hours for non-transferring torrents

Rename main directories of metafiles

Rename main directories of metafiles (torrent files) to make their naming more meaningful:

  • .temp -> .queue
  • .queue -> .downloading
  • .torrent -> .completed

Fix bug with modifying slots

Slots of a torrent (in both special and other group) didn't get modified in the following cases:

  • torrent was stopped before restarting rtorrent
  • torrent wasn't finished before restarting rtorrent

Solution: use event.download.resumed event to handle all the cases.

Modify how delete-queue is handled

Previously (see #16) the setup used the delqueue custom variable flag to handle .delqueue directory.
Change it to be handled by using only 1 custom variable unsafe_data that can have the following values: (safe-data) ,1(unsafe-data) ,2 (in delete-queue).

  1. Remove d.custom=delqueue custom variable flag completely.
  2. Modify d.remove_from_delqueue method to be able to specify whether unsafe_data is set to 1 or not set at all.
  3. This change requires version 1.1-0.9.6 or above of rtorrent-ps-ch: chros73/rtorrent-ps-ch#15
  4. Modify existing d.move_to manual moving method to support this change:
  • it receives 3 arguments: dirname,specialgroup,unsafe_data
  • d.move_to=rotating,1,1 or d.move_to=rotating,1,2
  • d.move_to=fullseed,1,
  • d.move_to=unsafe,,1 or d.move_to=unsafe,,2
  • d.move_to=various,,

Migration instructions:
Run the following commands in order, they will care about the rest.

  • d.multicall2=main,d.fix_delqueue_flag= : updates the unsafe_data custom variable value for every torrent
  • d.multicall2=main,"d.custom.set=delqueue," : deletes the old delqueue custom variable flag for every torrent
  • session.save= : saves all the sessions

Add modified pyrocore config file

Add modified pyrocore config file config.ini and change rtorrent_rc directive to our new config path: ~/.rtorrent-config.rc, otherwise pyrocore utils won't work. (See #36)

Modify sorting Started view

Modify sorting Started view: use d.last_active method instead of the custom variable d.custom=last_active. It gives more accurate result when peers are connected.

Bind key to toggle through selectable themes in rtorrent config

Bind key | to toggle through selectable themes.

  • add couple of methods that contains the available/desirable themes, e.g. cfg.theme.0
  • use pyro.theme.selected string variable method to store the actual filename is being used
  • apply toggle logic in form of using branches (since we can't write recursive methods in rtorrent config)

Note: the latter means that if a new theme is added to /removed from the available selection then the toggle logic has to be modified as well.

Make these 3 themes available for toggling in the following order:

  • color_scheme256-happy_pastel.rc
  • color_scheme256-solarized_yellow.rc
  • color_scheme256-solarized_blue.rc

(Refers to: #21 , #22 , #45)

Add postfix config entry

Add postfix config entry cfg.postfix that can help to use different config files, scripts with different setups based on naming. It can be an empty string as well.
Refactor the necessary code part.

Modify views

  • sort tracker and pyrotorque views by asc tracker domain and desc downloaded date
  • use extended canvas by default with uploaded data view
  • swap progress20 and progress40 colors

Add networking tweaks to the setup

Add networking tweaks to the setup, including:

  • system wide settings
  • rtorrent config entries in the form of: network.receive_buffer.size and network.send_buffer.size

The latter 2 adjust the send and receive buffer size for socket. Disabled by default (0), this means the default is used by OS (you have to modify the system wide settings!!!) (send_buffer_size, receive_buffer_size)
Increasing buffer sizes may help reduce disk seeking, connection polling as more data is buffered each time the socket is written to. It will result higher memory usage (not by rtorrent process!).

These values has to be adjusted in the system sysctl.conf file as well.

Dynamically adjusts the global download rate and refactor existing code

Idea: adjust dynamically upload rate and other attributes to favor one special group of torrents over the rest of them

A. Scenario: all we want to do is seeding (unfortunately we have to download before it :) ). We want to distinguished between 2 groups: a special group and the rest of them.

  • special group: torrents that belong to sites where seeding back is pretty hard. So we want to favor this group in any way.
  • the rest: seeding back is pretty easy, so it's not so important to provide the best circumstances for them, but we still want to seed as many of them as we can.

Applying this to downloading is easy but not to seeding.

B. Realization:

  • we use multiple watch directories to specify which belongs to the special group: which doesn't belongs to the slowup throttle that belongs to the special group
  • we raise the default min_peers, max_peers and max_uploads values for the special group.
  • we set priority to high for the special group that will take care of the downloading part
  • we define a throttle group (named slowup) and set an inital value for it. The special group doesn't use it, the rest of the torrents will use this slowup throttle forever.
  • we want to adjust the upload rate of this 'slowup' throttle on the fly (with the help of a script) depending on the special group current upload rate and the current global upload rate
  • you can check the current up rate values using ^X and i= (or just taking a look at the status bar of rtorrent-PS-CH)
  1. Refactor existing adjust_throttle_slowup and regarding info method: put these into an external shell script (it doesn't affect performance).
    This info also will be visible in the status bar of the patched rtorrent-ps: chros73/rtorrent-ps-ch#4
  2. Dynamically adjusts the global download rate to always leave enough bandwidth to the 1st main group upload rate. It's good for async connection (e.g. ADSL) where upload speed can be slowed down if download speed is close to max.
    Run this every minute.

Modify moving torrent logic

Right now auto moving torrents (when download is finished or after a hashcheck) is working fine until a target data or meta file exists. It happens pretty rarely but in this case:

  • download will be finished
  • since target data dir/file exists rtorrent can't move the data to the target dir but all attributes are already updated, so seeding will be broken straight away with a message: Download registered as completed, but hash check returned unfinished chunks.

So, do NOT move data dir/file NOR metafile anywhere when the target of one of these exist (or both):

  • leave both data and meta file in original dir (manual interaction required later)

Fix issue with human readable time method

When a custom variable contains a string, e.g. -1d then hrf_time method fails.

Solution: check whether the value is greater than 0. If not then print the value itself instead of the hrf time.

Original ticket: #7

Fix small bugs

These didn't affect anything but they were bugs:

  • remove duplicated entry for ip command
  • remove duplicated entry for untied_dir method and remove the misunderstood parameter
  • do not use full full path in pyro.bin_dir and pyro.rtorrent_rc methods
  • comment out unused pyro_watchdog scheduling

Refactor rtorrent-ps config file and add new views into it

Refactor rtorrent-ps.rc config file to make it more meaningful and add/modify more comments.

Also add new default views: started, stopped, incomplete, hashing, leeching.

Rebind keys for our custom views:

  • < : datasize
  • > : uploadeddata
  • % : ratio
  • @ : category

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.