johndoee / autotorrent Goto Github PK

With the recent addition of qbittorrent and possibly migrating my client from rtorrent to qbittorrent, I was thinking an option for each client profile to have it's own store_path would be useful.

For example, I have these two clients:

[client-rt]
client = rtorrent
url = scgi://127.0.0.1:5000
label = autotorrent

[client-qbt]
client = qbittorrent
url = http://127.0.0.1:8080
username = qbt
password = somepassword
category = autotorrent

The issue comes in, if I want to re-load all my .torrents, I have to temporarily change the global store_path while I add torrents to qbittorrent, but then change store_path back when I add to rtorrent.

So in my mind the setting would look like:

[client-rt]
client = rtorrent
url = scgi://127.0.0.1:5000
label = autotorrent
store_path = /mnt/user/Torrent/rtorrent

Probably an edge case, but thought some might find such a feature useful :)

i'm trying to run this script on about 40,000 torrents that ive gathered over the years and from different storage solutions, change in clients, and restoring from backups, etc.
i'm runnig into quite a few errors when running autotorrent.

This error seems to occur when the torrent file is corrupted and it completely stops the script. Removing the torrent and then re-running the script works past this point.

Traceback (most recent call last):
  File "autotorrent-env/bin/autotorrent", line 9, in <module>
    load_entry_point('autotorrent==1.7.1', 'console_scripts', 'autotorrent')()
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/cmd.py", line 210, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 500, in handle_torrentfile
    torrent = self.open_torrentfile(path)
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 568, in open_torrentfile
    return bdecode(f.read())
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/bencode.py", line 67, in bdecode
    raise BTFailure("invalid bencoded value (data after valid prefix)")
autotorrent.bencode.BTFailure: invalid bencoded value (data after valid prefix)

Sometimes there exists a duplicate torrent right next to each other which throws the error below. This also stops the script but i can re-run it without having to delete the duplicate torrent file.

 [OK]        '***************************************' Torrent added successfully
Traceback (most recent call last):
  File "autotorrent-env/bin/autotorrent", line 9, in <module>
    load_entry_point('autotorrent==1.7.1', 'console_scripts', 'autotorrent')()
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/cmd.py", line 210, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 549, in handle_torrentfile
    if self.client.add_torrent(torrent, destination_path, files['files'], fast_resume):
  File "/autotorrent-env/lib/python2.7/site-packages/autotorrent/clients/deluge.py", line 148, in add_torrent
    'seed_mode': fast_resume})
  File "/autotorrent-env/lib/python2.7/site-packages/deluge_client/client.py", line 253, in call
    return self._receive_response(self.deluge_version, self.deluge_protocol_version)
  File "/autotorrent-env/lib/python2.7/site-packages/deluge_client/client.py", line 232, in _receive_response
    raise exception(exception_msg)
deluge_client.client.AddTorrentError: Torrent already in session (*********************).
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/deluge/core/rpcserver.py", line 326, in dispatch
    ret = self.factory.methods[method](*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/deluge/core/core.py", line 483, in add_torrent_file
    return self.torrentmanager.add(
  File "/usr/lib/python3.9/site-packages/deluge/core/torrentmanager.py", line 536, in add
    __, add_torrent_params = self._build_torrent_params(
  File "/usr/lib/python3.9/site-packages/deluge/core/torrentmanager.py", line 446, in _build_torrent_params
    raise AddTorrentError('Torrent already in session (%s).' % torrent_id)
deluge.error.AddTorrentError: Torrent already in session (*********************)

also this error seems to be triggered with the Ω symbol in a torrent filename.

Traceback (most recent call last):
File "/mnt/user/appdata/autotorrent/autotorrent-env/bin/autotorrent", line 11, in
sys.exit(commandline_handler())
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/cmd.py", line 212, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 453, in handle_torrentfile
self.link_files(destination_path, files['files'])
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 342, in link_files
if not os.path.isdir(file_path):
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib64/python2.7/genericpath.py", line 49, in isdir
st = os.stat(s)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u03a9' in position 95: ordinal not in range(128)

This error is when the script tries to read a torrent file with no contents i.e 0 byte file size.

Traceback (most recent call last):
File "/mnt/user/appdata/autotorrent/autotorrent-env/bin/autotorrent", line 11, in
sys.exit(commandline_handler())
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/cmd.py", line 212, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 419, in handle_torrentfile
torrent = self.open_torrentfile(path)
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/at.py", line 487, in open_torrentfile
return bdecode(f.read())
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/autotorrent/bencode.py", line 62, in bdecode
r, l = decode_func[six.indexbytes(x, 0)](x, 0)
File "/mnt/user/appdata/autotorrent/autotorrent-env/lib/python2.7/site-packages/six.py", line 666, in indexbytes
return ord(buf[i])
IndexError: string index out of range

Would be really nice if we could just skip these files and then let the script keep running.

I was reading the working modes on the main readme and realized that there is a particular use case that would be very useful in being added. Some trackers, for some reason, have slightly altered root folder and/or filenames, but will otherwise hash check as an exact match.

The strings are only a few characters off, for example tracker A's torrent will download as 'hello.there.2005.1080p.WEB-DL.x264.AAC-BoldOne.mkv' and tracker B's will download as 'hello.there.2005.1080p.WEBDL.x264.AAC-BoldOne.mkv'

The dash is missing from WEB-DL in tracker B's download. I understand the hash_size working mode could be used against this, but seeing as these strings only have a small hamming distance of differences, it could be worked into the Normal mode in order to avoid hash checking the files.

Thank you for reading.

INFO:autotorrent:Handling file /home/btn/wcd/Bill Evans Trio - Explorations - 2000 (CD - MP3 - 320)-30036626.torrent
INFO:autotorrent:Found name u'Bill Evans - Explorations [320]' for torrent
Traceback (most recent call last):
File "./autotorrent", line 9, in
load_entry_point('autotorrent==1.6.0', 'console_scripts', 'autotorrent')()
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/cmd.py", line 214, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 417, in handle_torrentfile
found_size, missing_size, files = self.parse_torrent(torrent)
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 308, in parse_torrent
files = self.index_torrent(torrent)
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 232, in index_torrent
orig_path = [x.decode('utf-8') for x in f[b'path'] if x] # remove empty fragments
File "/home/mgmt/autotorrent-env/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbf in position 27: invalid start byte

This is more of a statement not a question
depending on your setup it seems deluge might not be good at picking up every torrent yet

I'd like to try with the stable channel to see if I can get the same results.

The effect is that qbittorrent is able to seed way more torrents then deluge. The error below results in those torrents not being added. If though they are 100 percent in the database.

    filedump=filedump, options=options, filename=filename, save_state=save_state
  File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.15-py2.7.egg/deluge/core/torrentmanager.py", line 364, in add
    torrent_info.rename_file(index, fname.encode('utf8'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 172: ordinal not in range(128)

With scan modes now separately supporting incomplete files with exact filename, and complete files with different filename but same size, it opens the door to bypassing files that are close to 100% and have a different filename.

As an advanced search tool by using a command-line argument, tell autotorrent to scan all chunks, regardless of filename or file size.

Presumably, this would be run on very specific subfolders containing data for a single torrent.

I am trying to migrate sever hundred torrents to another client. About half of them work. The other half say the files are missing but I have verified they are not. I can still access the files and the old torrent client successfully completes a hash check. Any idea what's going on?

When running autotorrent -a /mnt/user/General/Torrs/*.torrent, after a recent rtorrent Docker update, I am getting the following. What do I need to do?

  File "/usr/bin/autotorrent", line 11, in <module>
    load_entry_point('autotorrent==1.6.3', 'console_scripts', 'autotorrent')()
  File "/usr/lib64/python3.6/site-packages/autotorrent/cmd.py", line 210, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/usr/lib64/python3.6/site-packages/autotorrent/at.py", line 465, in handle_torrentfile
    if self.client.add_torrent(torrent, destination_path, files['files'], fast_resume):
  File "/usr/lib64/python3.6/site-packages/autotorrent/clients/rtorrent.py", line 179, in add_torrent
    self.proxy.load_start(*cmd)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.6/site-packages/autotorrent/scgitransport.py", line 67, in single_request
    return self.parse_response(response_body)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1342, in parse_response
    return u.close()
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 656, in close
    raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -506: "Method 'load_start' not defined">```

I found a bug I believe
I thought it had to do with the new version and changes that occured in correcting the debugging
but it seems to be with deluged and the if a label already exist

Deluge
Found 1 torrent(s)
Traceback (most recent call last):
File "autotorrent-env/bin/autotorrent", line 11, in
sys.exit(commandline_handler())
File "/home/snickerpop/autotorrent/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/cmd.py", line 210, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/home/snickerpop/autotorrent/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 468, in handle_torrentfile
if self.client.add_torrent(torrent, destination_path, files['files'], fast_resume):
File "/home/snickerpop/autotorrent/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/clients/deluge.py", line 151, in add_torrent
self.rpcclient.call('label.add', self.label)
File "/home/snickerpop/autotorrent/autotorrent-env/local/lib/python2.7/site-packages/deluge_client/client.py", line 214, in call
return self._receive_response(self.deluge_version)
File "/home/snickerpop/autotorrent/autotorrent-env/local/lib/python2.7/site-packages/deluge_client/client.py", line 193, in _receive_response
raise exception(exception_msg)
deluge_client.client.WrappedException: Label already exists, Exception, Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/deluge-1.3.15-py2.7.egg/deluge/core/rpcserver.py", line 302, in dispatch
ret = self.factory.methods[method](*args, **kwargs)
File "build/bdist.linux-x86_64/egg/deluge/plugins/label/core.py", line 174, in add
check_input(not (label_id in self.labels), _('Label already exists'))
File "build/bdist.linux-x86_64/egg/deluge/plugins/label/core.py", line 68, in check_input
raise Exception(message)
Exception: Label already exists

The program is still works
but if your doing something like
2>> or just want to see from the commandline the okay torrents. They are all missing and it only has
found torrent message, and nothing about which torrent was found

For now the fix seems to be removing the label as it is not needed for deluge to run.

HTTPSConnectionPool(host='127.0.0.1', port=9093): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_record', 'wrong version number')],)",),))

SSL is given via letsencrypt
URL = url = https://name:[email protected]:9093

I have a many-device array. I'm interested in using this code to cross-seed files, but having one store_path won't work, as I get OSError: [Errno 18] Invalid cross-device link.

The way my system is setup, is each hard drive has it's own slot under /mnt/diskX, so for example, if I want to ln files from /mnt/disk5, /mnt/disk5 has to be the destination, not a parent folder like /mnt/disks. Is it already possible to do that with this application in the config, and if not, can it be coded in? I'm willing to pay for this feature.

A way to do the scan and the linking without automatically adding it to the client.

Use-case: scanning on the home system to find out what to upload to the seedbox.

In a large library, full rebuild of the database will take a long time each time it's re-built.

It would be awesome if there is a way to also store last changed timestamp in the db, and only update the hashes of files changed in the next re-build.

I think this will require a db layer change, so might be difficult.

This scanmode tries to find files by looking for exact filesize and then use piece hashes to verify the file is correct.

Criteria:

• Exact filesize
• File must be self-contained with a piece (i.e. a whole piece must be within the file, so won't find small files)

The same bug referenced in issues #20 and #21. autotorrent -r seems to have a hard time handling the unicode in my downloads folder.

I'm having issues getting this to run on the new version of ubuntu

The problem seems to be with the socket file. For some reason autotorrent doesn't seem to be able to connect to any rtorrent instanes

I went back to python3.6, but the issue still persist
I

File "autotorrent-env/bin/autotorrent", line 11, in <module> load_entry_point('autotorrent==1.7.1', 'console_scripts', 'autotorrent')() File "/home/main/autotorrent/autotorrent-env/lib/python3.6/site-packages/autotorrent/cmd.py", line 207, in commandline_handler at.populate_torrents_seeded() File "/home/main/autotorrent/autotorrent-env/lib/python3.6/site-packages/autotorrent/at.py", line 84, in populate_torrents_seeded self.torrents_seeded = set(x.lower() for x in self.client.get_torrents()) File "/home/main/autotorrent/autotorrent-env/lib/python3.6/site-packages/autotorrent/clients/rtorrent.py", line 122, in get_torrents return set(x.lower() for x in self.proxy.download_list()) File "/usr/lib/python3.6/xmlrpc/client.py", line 1112, in __call__ return self.__send(self.__name, args) File "/usr/lib/python3.6/xmlrpc/client.py", line 1452, in __request verbose=self.__verbose File "/usr/lib/python3.6/xmlrpc/client.py", line 1154, in request return self.single_request(host, handler, request_body, verbose) File "/home/main/autotorrent/autotorrent-env/lib/python3.6/site-packages/autotorrent/scgitransport.py", line 42, in single_request s.connect(self.socket_path) FileNotFoundError: [Errno 2] No such file or directory

Hi,
Is there a way to force add torrents even if's already been added?
For example, if I add a torrent and remove it from the client without selecting remove + data I can no longer add it again.
If there were a few torrents I could remove the links manually but in the case of 100's it would take hours.

When trying to build the initial database it just gets stuck on the "Rebuilding database" message. I left it for well over an hour, which should be more than enough time. The database file never increases in size. I'm not sure what's going on. I am using a mapped network drive on WSL, so I'm sure it's something to do with that. I can browse the drive (/mnt/z) just fine. Sudo didn't help. Any ideas?

Python3 development and mainline:
Rebuilding database
Traceback (most recent call last):
File "autotorrent-env/bin/autotorrent", line 8, in
sys.exit(commandline_handler())
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/cmd.py", line 197, in commandline_handler
db.rebuild()
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 159, in rebuild
self.insert_into_database(root, f, 'normal')
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 49, in insert_into_database
self._insert_into_database(root, f, mode, prefix, unsplitable_name)
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 86, in _insert_into_database
key = self.keyify(size, normalized_filename)
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 282, in keyify
return hashlib.sha256(key.encode('utf-8')).hexdigest()
UnicodeEncodeError: 'utf-8' codec can't encode character '\udca0' in position 43: surrogates not allowed

Python2 development and mainline:
Rebuilding database
Traceback (most recent call last):
File "autotorrent-env/bin/autotorrent", line 8, in
sys.exit(commandline_handler())
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/cmd.py", line 197, in commandline_handler
db.rebuild()
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 159, in rebuild
self.insert_into_database(root, f, 'normal')
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 49, in insert_into_database
self._insert_into_database(root, f, mode, prefix, unsplitable_name)
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 86, in _insert_into_database
key = self.keyify(size, normalized_filename)
File "/home/gentoo/autotorrent-env/lib/python3.7/site-packages/autotorrent/db.py", line 282, in keyify
return hashlib.sha256(key.encode('utf-8')).hexdigest()
UnicodeEncodeError: 'utf-8' codec can't encode character '\udca0' in position 43: surrogates not allowed

In #16
I would not mind being limited to python2.7 if I could use this.

I tried copying at.py over but it seems the branches have diverged too far.

I have tried downloading and building all locales and have my main set at utf8en

Master branch is becoming rather outdated. Recommend updating it.

I think it'd be great if this program had a way to handle matches that are in-progress. Currently it would be up to the user to realize that a match would be found if only the in-progress download were finished, then it's up to them to retry it once it's finished.

Imagine a scenario where a match is found by file name(s) but not by size because it's in-progress. It would be amazing if instead of outright not matching, it were somehow queued to be added once the download did finish (perhaps pending a re-check that it still is a match).

EDIT: At the very least it would be extremely useful if autotorrent at least notified the user of this, i.e. that a torrent would match once an in-progress download completed, then users can handle that however they see fit.

One way to do that would be to poll rtorrent at intervals to determine when the download finishes, requiring auto-torrent to remain running in the meantime.

Another more sophisticated way might be to register some kind of on_finished event handler/method with rtorrent. I'm not sure if it's possible to specify handlers per-torrent, maybe they're only global, but in that case it can be registered so that it only executes if the infohash matches the in-progress/waited-upon torrent with something like:

# register a global on_finished handler which says:
# if the finished download's infohash matches the one we're waiting on, load its cross-seed
`branch=equal=$d.get_hash=,HARDCODEDHASH,\"load=path/to.torrent\"

I'm not sure if equal= is a method, this is just a rough sketch of what can probably be done. So again, a partial match would be found (file name but size < target_size, we'd then get the infohash corresponding to that download and register that handler on_finished. We'd come to know the infohash corresponding to that match by getting the rtorrent state at launch with a d.multicall, fetching the infohash, the download's directory/path, etc. Then we'd join the match on the directory/path to arrive at the infohash of the match.

EDIT: Added note above.

I can't get any torrents to add to my client (rtorrent). Grabbing the torrent file that autotorrent created (__tmp_torrent61aa17eb-490c-4d22-8c6b-fec0a5470b66.torrent) and adding it in rutorrent's GUI worked fine.

Verbose output below:

adamwinn@DiskStation:~/scripts/autotorrent$ autotorrent -a ubuntu-19.04-desktop-amd64.iso.torrent --verbose
/usr/local/lib/python2.7/site-packages/cffi/model.py:526: UserWarning: 'point_conversion_form_t' has no values explicitly defined; next version will refuse to guess which integer type it is meant to be (unsigned/signed, int/long)
  % self._get_c_name())
DEBUG:autotorrent.clients.rtorrent:Creating Normal XMLRPC Proxy with url 'http://172.17.0.1:51414/RPC2'
Found 1 torrent(s)
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
INFO:autotorrent:Handling file /volume1/homes/adamwinn/scripts/autotorrent/ubuntu-19.04-desktop-amd64.iso.torrent
DEBUG:autotorrent:Handling torrent name 'ubuntu-19.04-desktop-amd64.iso'
INFO:autotorrent:Found name u'ubuntu-19.04-desktop-amd64.iso' for torrent
DEBUG:autotorrent.db:Keyify: 2097152000|ubuntu-19.04-desktop-amd64.iso
INFO:autotorrent:Preparing torrent using link mode
DEBUG:autotorrent:Making soft link from '/var/services/homes/adamwinn/scripts/autotorrent/files/ubuntu-19.04-desktop-amd64.iso' to u'./ubuntu-19.04-desktop-amd64.iso/ubuntu-19.04-desktop-amd64.iso'
INFO:autotorrent.clients.rtorrent:Trying to add a new torrent to rtorrent: 'ubuntu-19.04-desktop-amd64.iso'
INFO:autotorrent.clients.rtorrent:Trying to do fast resume data
DEBUG:autotorrent.clients.rtorrent:Handling file {u'path': [u'ubuntu-19.04-desktop-amd64.iso'], u'length': 2097152000, u'completed': True, u'actual_path': '/var/services/homes/adamwinn/scripts/autotorrent/files/ubuntu-19.04-desktop-amd64.iso'}
INFO:autotorrent.clients.rtorrent:This torrent is complete, setting bitfield to chunk count
INFO:autotorrent.clients.rtorrent:Sending to rtorrent: ['/volume1/homes/adamwinn/scripts/autotorrent/ubuntu-19.04-desktop-amd64.iso/__tmp_torrent4b43b543-c209-48ad-9e0e-b06054ec656c.torrent', 'd.directory_base.set="/volume1/homes/adamwinn/scripts/autotorrent/ubuntu-19.04-desktop-amd64.iso"', 'd.custom1.set=autotorrent']
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
INFO:autotorrent.clients.rtorrent:Getting a list of torrent hashes
WARNING:autotorrent.clients.rtorrent:Torrent was not added to rtorrent within reasonable timelimit
 [Failed]    'ubuntu-19.04-desktop-amd64.iso' Failed to send torrent to client

I gave an invalid path for disk1 in autotorrent.conf and no error was thrown when I rebuilt the database

scan error's out when hitting system volume.

Traceback (most recent call last):
File "./autotorrent", line 11, in
sys.exit(commandline_handler())
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/cmd.py", line 113, in commandline_handler
db.rebuild()
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/db.py", line 156, in rebuild
self.insert_into_database(root, f, 'normal')
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/db.py", line 49, in insert_into_database
self._insert_into_database(root, f, mode, prefix, unsplitable_name)
File "/home/mgmt/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/db.py", line 67, in _insert_into_database
size = os.path.getsize(path)
File "/home/mgmt/autotorrent-env/lib/python2.7/genericpath.py", line 49, in getsize
return os.stat(filename).st_size
OSError: [Errno 13] Permission denied: '/mnt/hgfs/D/System Volume Information'

could add the aria2 compatibility using the lib https://pypi.org/project/aria2p/

I tried matching a DVD and it symlinked to parts of 7 other DVDs, instead of the correct DVD. Flag in the Mist is the title of the DVD

lrwxrwxrwx  1 xxx users  54 Jun 10 19:58 VIDEO_TS.BUP -> /volume1/LTS-Movies-MZ/THESPHINX/VIDEO_TS/VIDEO_TS.BUP
lrwxrwxrwx  1 xxx users  54 Jun 10 19:58 VIDEO_TS.IFO -> /volume1/LTS-Movies-MZ/THESPHINX/VIDEO_TS/VIDEO_TS.IFO
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VIDEO_TS.VOB -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VIDEO_TS.VOB
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VTS_01_0.BUP -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VTS_01_0.BUP
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VTS_01_0.IFO -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VTS_01_0.IFO
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VTS_01_0.VOB -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VTS_01_0.VOB
lrwxrwxrwx  1 xxx users  73 Jun 10 19:58 VTS_01_1.VOB -> /volume1/LTS-Movies-MZ/The.Lost.Prince.2003.DVD.PAL/VIDEO_TS/VTS_01_1.VOB
lrwxrwxrwx  1 xxx users  73 Jun 10 19:58 VTS_01_2.VOB -> /volume1/LTS-Movies-MZ/The.Lost.Prince.2003.DVD.PAL/VIDEO_TS/VTS_01_2.VOB
lrwxrwxrwx  1 xxx users  73 Jun 10 19:58 VTS_01_3.VOB -> /volume1/LTS-Movies-MZ/The.Lost.Prince.2003.DVD.PAL/VIDEO_TS/VTS_01_3.VOB
lrwxrwxrwx  1 xxx users  71 Jun 10 19:58 VTS_01_4.VOB -> /volume1/LTS-Movies-MZ/The Brotherhood V - Alumni/VIDEO_TS/VTS_01_4.VOB
lrwxrwxrwx+ 1 xxx users 111 Jun 10 19:58 VTS_01_5.VOB -> /volume1/LTS-Movies-MZ/The Woody Woodpecker and Friends Classic Cartoon Collection/Disc 3/VIDEO_TS/VTS_01_5.VOB
lrwxrwxrwx+ 1 xxx users 111 Jun 10 19:58 VTS_01_6.VOB -> /volume1/LTS-Movies-MZ/The Woody Woodpecker and Friends Classic Cartoon Collection/Disc 3/VIDEO_TS/VTS_01_6.VOB
lrwxrwxrwx+ 1 xxx users  62 Jun 10 19:58 VTS_01_7.VOB -> /volume1/LTS-Movies-MZ/Young People 1972/VIDEO_TS/VTS_01_7.VOB
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VTS_01_8.VOB -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VTS_01_8.VOB
lrwxrwxrwx  1 xxx users 110 Jun 10 19:58 VTS_02_0.BUP -> /volume1/LTS-Movies-MZ/The Cartoonist - Jeff Smith, BONE and the Changing Face of Comics/VIDEO_TS/VTS_02_0.BUP
lrwxrwxrwx  1 xxx users 110 Jun 10 19:58 VTS_02_0.IFO -> /volume1/LTS-Movies-MZ/The Cartoonist - Jeff Smith, BONE and the Changing Face of Comics/VIDEO_TS/VTS_02_0.IFO
lrwxrwxrwx  1 xxx users  59 Jun 10 19:58 VTS_02_0.VOB -> /volume1/LTS-Movies-MZ/REMOTE_VIEWING/VIDEO_TS/VTS_02_0.VOB
lrwxrwxrwx+ 1 xxx users  96 Jun 10 19:58 VTS_02_1.VOB -> /volume1/LTS-Movies-AL/Flag in the Mist (1965) R2J NTSC DVD9 - Custom subs/VIDEO_TS/VTS_02_1.VOB

Following the steps in the readme, I get:

user@server460:~/.local/bin$ autotorrent-env/bin/pip --create_config autotorrent-env/autotorrent.conf

Usage:   
  pip <command> [options]

no such option: --create_config

Am I missing a step? Or can you provide what a sample config file looks like and I can just create it myself?

I have 2 torrents in my client and one is paused and one is stopped. I ran the script and it said both were seeded. A check should be made for paused or stopped, or the verbiage should be reworded

Most old content and content that has been curated in to a Library has been renamed, subtracted from, and added to.

To use autotorrent on these files, it would be necessary to check the contents in full using piece hashes.
Worst-case scenario, some files are missing or some of the data within the files is missing, but if there is a peer who has the 3% that you are missing, and you have the other 97%, then both could have 100% by 'seeding' the incomplete torrent.

`Traceback (most recent call last):
File "autotorrent-env/bin/autotorrent", line 8, in
sys.exit(commandline_handler())
File "/root/autotorrent-env/lib/python3.8/site-packages/autotorrent/cmd.py", line 210, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/root/autotorrent-env/lib/python3.8/site-packages/autotorrent/at.py", line 500, in handle_torrentfile
torrent = self.open_torrentfile(path)
File "/root/autotorrent-env/lib/python3.8/site-packages/autotorrent/at.py", line 568, in open_torrentfile
return bdecode(f.read())
File "/root/autotorrent-env/lib/python3.8/site-packages/autotorrent/bencode.py", line 62, in bdecode
r, l = decode_func[six.indexbytes(x, 0)](x, 0)
IndexError: index out of range

due to an empty (zero bytes) torrent file

Can we utilize one of these methods
surrogatepass, replaces, surrogateescape

https://docs.python.org/3/library/codecs.html

I ended up using surrogatepass, and it seems to have fix the issue
at the very least the program doesn't crash

I have your script setup and it seems to work perfectly other than the fact that torrents added to rtorrent always get set to Paused after a few seconds of seeding. Config is below.

[general]
db = _autotorrent.db
ignore_files =
store_path = store_paths/seeds/
add_limit_size = 64000000
add_limit_percent = 1
link_type = soft
scan_mode = unsplitable,normal

[disks]
disk1 = /media/dmm/spicywiener/private/rtorrent/data/TV
disk2 = /media/dmm/spicywiener/private/rtorrent/data/Movies
disk3 = ~/pip/bin/store_paths/seeds/

[client]
url = scgi:///media/dmm/spicywiener/private/rtorrent/.socket
client = rtorrent
label = autotorrent

Not sure what is causing this, but I'm getting this on some .torrent files:

Traceback (most recent call last):
  File "/usr/bin/autotorrent", line 11, in <module>
    load_entry_point('autotorrent==1.6.2', 'console_scripts', 'autotorrent')()
  File "/usr/lib64/python3.6/site-packages/autotorrent/cmd.py", line 214, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/usr/lib64/python3.6/site-packages/autotorrent/at.py", line 419, in handle_torrentfile
    torrent = self.open_torrentfile(path)
  File "/usr/lib64/python3.6/site-packages/autotorrent/at.py", line 484, in open_torrentfile
    return bdecode(f.read())
  File "/usr/lib64/python3.6/site-packages/autotorrent/bencode.py", line 62, in bdecode
    r, l = decode_func[six.indexbytes(x, 0)](x, 0)
IndexError: index out of range

This happens if the torrent contains an empty files (found_size and missing_size = 0)

Traceback (most recent call last):
  File "autotorrent-env/bin/autotorrent", line 11, in <module>
    load_entry_point('autotorrent==1.6.1', 'console_scripts', 'autotorrent')()
  File "/var/lib/deluge/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/cmd.py", line 214, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/var/lib/deluge/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 430, in handle_torrentfile
    missing_percent = (missing_size / (found_size + missing_size)) * 100
ZeroDivisionError: division by zero

Having issues with this on raspberry pi2. All dependencies appear to be satisfied.

pi@raspberrypi ~ $ virtualenv autotorrent-env
New python executable in autotorrent-env/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
pi@raspberrypi ~ $ autotorrent-env/bin/pip install git+https://github.com/JohnDoee/autotorrent.git#develop
Downloading/unpacking git+https://github.com/JohnDoee/autotorrent.git
Cloning https://github.com/JohnDoee/autotorrent.git to /tmp/pip-g_C5bx-build
Running setup.py egg_info for package from git+https://github.com/JohnDoee/autotorrent.git

Downloading/unpacking six (from autotorrent==1.6.0)
Downloading six-1.10.0.tar.gz
Running setup.py egg_info for package six

no previously-included directories found matching 'documentation/_build'

Downloading/unpacking deluge-client (from autotorrent==1.6.0)
Downloading deluge-client-1.0.3.tar.gz
Running setup.py egg_info for package deluge-client

Downloading/unpacking requests (from autotorrent==1.6.0)
Downloading requests-2.9.1.tar.gz (484Kb): 484Kb downloaded
Running setup.py egg_info for package requests

Installing collected packages: six, deluge-client, requests, autotorrent
Running setup.py install for six

no previously-included directories found matching 'documentation/_build'

Running setup.py install for deluge-client

Running setup.py install for requests

Running setup.py install for autotorrent

Installing autotorrent script to /home/pi/autotorrent-env/bin

Successfully installed six deluge-client requests autotorrent
Cleaning up...
pi@raspberrypi ~ $ pip freeze
acdcli==0.3.1
appdirs==1.4.0
autotorrent==1.6.0
chardet==2.0.1
colorama==0.3.5
deluge-client==1.0.3
mcpi==0.1.1
numpy==1.6.2
picamera==1.10
pifacecommon==4.1.2
pifacedigitalio==3.0.4
pygame==1.9.2a0
pyOpenSSL==0.13.1
pyserial==2.5
python-dateutil==2.4.2
requests==2.9.1
requests-toolbelt==0.5.1
RPi.GPIO==0.5.11
six==1.10.0
SQLAlchemy==1.0.11

pi@raspberrypi ~ $ cd autotorrent-env/bin/

pi@raspberrypi ~/autotorrent-env/bin $ source activate
(autotorrent-env)pi@raspberrypi ~/autotorrent-env/bin $ pip3 freeze
acdcli==0.3.1
appdirs==1.4.0
autotorrent==1.6.0
chardet==2.0.1
colorama==0.3.5
deluge-client==1.0.3
mcpi==0.1.1
numpy==1.6.2
picamera==1.10
pifacecommon==4.1.2
pifacedigitalio==3.0.4
pygame==1.9.2a0
pyOpenSSL==0.13.1
pyserial==2.5
python-dateutil==2.4.2
requests==2.9.1
requests-toolbelt==0.5.1
RPi.GPIO==0.5.11
six==1.10.0
SQLAlchemy==1.0.11

(autotorrent-env)pi@raspberrypi ~/autotorrent-env/bin $ cp ~/_autotorrent.db ~/autotorrent.conf .

(autotorrent-env)pi@raspberrypi ~/autotorrent-env/bin $ ./autotorrent -t
Connected to torrent client successfully!
result: cwd:'/home/pi', pid:18437

(autotorrent-env)pi@raspberrypi ~/autotorrent-env/bin $ ./autotorrent -a ~/ahd/Blood\ Ties\ -\ 2013\ (Blu-ray\ -\ MKV\ -\ h.264\ Remux).torrent
Found 1 torrent(s)
Traceback (most recent call last):
File "./autotorrent", line 9, in
load_entry_point('autotorrent==1.6.0', 'console_scripts', 'autotorrent')()
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/cmd.py", line 214, in commandline_handler
result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 421, in handle_torrentfile
if self.check_torrent_in_client(torrent):
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 476, in check_torrent_in_client
info_hash = self.get_info_hash(torrent)
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/at.py", line 90, in get_info_hash
return hashlib.sha1(bencode(torrent[b'info'])).hexdigest()
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/bencode.py", line 122, in bencode
encode_func[type(x)](x, r)
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/bencode.py", line 102, in encode_dict
encode_func[type(v)](v, r)
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/bencode.py", line 95, in encode_list
encode_func[type(i)](i, r)
File "/home/pi/autotorrent-env/local/lib/python2.7/site-packages/autotorrent/bencode.py", line 102, in encode_dict
encode_func[type(v)](v, r)
KeyError: <type 'long'>
(autotorrent-env)pi@raspberrypi ~/autotorrent-env/bin $

hi

i am checking / adding a lot of torrents right now and for some torrents i get the either the error message 13 or 17.
OSError 13: Permission denied.
OSError 17: File / Folder already existing (or something similar, forgot to screenshot it)
https://www.imagebanana.com/s/big/1725/jbLBSECo.jpg

Is there a way to skip these errors?
Right now i have to delete the *.torrent, which causes the Problem and start the import again.

greetings Stumpy
and ty for your great work :)

The Readme lists the creation command as autotorrent-env/bin/pip but it should be autotorrent-env/bin/autotorrent

I can't build my initial database -- i get this error:

autotorrent-env/bin/autotorrent -r


Rebuilding database
Traceback (most recent call last):
  File "autotorrent-env/bin/autotorrent", line 11, in <module>
    sys.exit(commandline_handler())
  File "/home/user/temp/autotorrent-env/lib/python3.5/site-packages/autotorrent/cmd.py", line 197, in commandline_handler
    db.rebuild()
  File "/home/user/temp/autotorrent-env/lib/python3.5/site-packages/autotorrent/db.py", line 163, in rebuild
    self.insert_into_database(root, f, 'exact', 'f')
  File "/home/user/temp/autotorrent-env/lib/python3.5/site-packages/autotorrent/db.py", line 49, in insert_into_database
    self._insert_into_database(root, f, mode, prefix, unsplitable_name)
  File "/home/user/temp/autotorrent-env/lib/python3.5/site-packages/autotorrent/db.py", line 63, in _insert_into_database
    key = self.keyify(prefix, f)
  File "/home/user/temp/autotorrent-env/lib/python3.5/site-packages/autotorrent/db.py", line 282, in keyify
    return hashlib.sha256(key.encode('utf-8')).hexdigest()
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 6: surrogates not allowed`

ls | grep * 1 2

Running autotorrent with dry run
`Found 2 torrent(s)
Torrent: /home/xxx/1.torrent
Found data: 0.0 bytes - Missing data: 75.4 GB - Would add: No
Local files used:

Torrent: /home/xxx/2.torrent
Found data: 0.0 bytes - Missing data: 42.0 GB - Would add: No
Local files used:
`
The torrents are exactly the same as the ones in the folder. This doesn't like RAW discs. When I tested with a single file, it worked perfectly.

autotorrent.conf
`[general]
db = MountDB
ignore_files =
store_path = /home/xxx/links/
add_limit_size = 64000000
add_limit_percent = 1
link_type = soft
scan_mode = exact

[disks]
disk1=/home/xxx/mount/`

Sending to qBittorrent

Part of the unicode problems stem from still having Python 2 support, this issue is all those issues folded into one.

Related issues: #16 #20 #35 #42

• Move to pathlib
• Move from shelve to sqlite as internal database

It would be very nice if adding a torrent returned an exit code indicative of whether or not it was added, so we could do something based on whether it was added or not.

It seems like the -a option accepts one or more files, so perhaps it would only make sense if a single path was passed, which is how I personally use it anyway.

awinn@Adams-MBP ~/D/c/a/autotorrent> autotorrent-env/bin/autotorrent -a ~/Downloads/Snatches/*.torrent
Failed to execute process 'autotorrent-env/bin/autotorrent'. Reason:
The total size of the argument and environment lists 1.3MB exceeds the operating system limit of 256kB.
Try running the command again with fewer arguments.

The script should be written in such a way where we pass a dir name, instead of *.torrent, to prevent this

DEBUG:autotorrent.clients.transmission:Calling 'torrent-add' args {'download-dir': '/var/lib/transmission/Downloads', 'paused': True, 'metainfo': b'<snip>'}
Traceback (most recent call last):
  File "/home/user/tmp/autotorrent/autotorrent-env/bin/autotorrent", line 11, in <module>
    sys.exit(commandline_handler())
  File "/home/user/tmp/autotorrent/autotorrent-env/lib/python3.5/site-packages/autotorrent/cmd.py", line 127, in commandline_handler
    result = at.handle_torrentfile(os.path.join(current_path, torrent), dry_run)
  File "/home/user/tmp/autotorrent/autotorrent-env/lib/python3.5/site-packages/autotorrent/at.py", line 454, in handle_torrentfile
    if self.client.add_torrent(torrent, destination_path, files['files'], fast_resume):
  File "/home/user/tmp/autotorrent/autotorrent-env/lib/python3.5/site-packages/autotorrent/clients/transmission.py", line 98, in add_torrent
    result = self.call('torrent-add', **kwargs)
  File "/home/user/tmp/autotorrent/autotorrent-env/lib/python3.5/site-packages/autotorrent/clients/transmission.py", line 46, in call
    r = self._call(method, **kwargs)
  File "/home/user/tmp/autotorrent/autotorrent-env/lib/python3.5/site-packages/autotorrent/clients/transmission.py", line 38, in _call
    result = requests.post(self.url, data=json.dumps({'method': method, 'arguments': kwargs}), headers={'X-Transmission-Session-Id': self._session_id})
  File "/usr/lib64/python3.5/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib64/python3.5/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python3.5/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python3.5/json/encoder.py", line 180, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: b'<snip>' is not JSON serializable

The solution is to change lib/python3.5/site-packages/autotorrent/clients/transmission.py: encoded_torrent -> encoded_torrent.decode('utf-8') and name -> name.decode('utf-8')

Not sure how that change will work with python2.

Native support for qbittorent would be nice

Right now I just made a secondary fake rtorrent client so the program will run, and create the systemlinks.

I'm planning on the adding the torrents manually through qbittorent, and maybe skipping hash check

This scan mode tries to find files with an exact filename but a different filesize. The filesize can be different due something breaking the file. It is expected that the pieces can be aligned from either the start or the end of the file to ensure it is the correct file.

The mode will delete from the beginning or the end of the file to align it with the expected filesize.

Criteria

• Exact filename
• File must be self-contained with a piece (i.e. a whole piece must be within the file, so won't find small files)
• File is only broken from the beginning or the end to make it possible to find a piece within the file.