nosmokingbandit / watcher3 Goto Github PK

After a nzbget that is marked for "failed" download, watcher3 don't pick up the next in line.

I use password dector script to check early on of a zip file is password protected. But after the marked bad of failed, its look like watcher3 doesn't know to pick something els.

Nzbget:

	Tue Apr 18 2017 21:57:14	Post-process-script watcher.py3 for Moon.2009.1080p.Bluray.x264.DTS-WiKi successful
INFO	Tue Apr 18 2017 21:57:14	watcher: Sending Moon.2009.1080p.Bluray.x264.DTS-WiKi to Watcher as Failed.
INFO	Tue Apr 18 2017 21:57:13	Executing post-process-script watcher.py3 for Moon.2009.1080p.Bluray.x264.DTS-WiKi

Wachter3:

INFO 2017-04-18 21:57:14,627 core.postprocessing.POST: #################################
INFO 2017-04-18 21:57:14,627 core.postprocessing.POST: Post-processing complete.
INFO 2017-04-18 21:57:14,627 core.postprocessing.POST: {
  "data": {
    "added_date": "2017-04-18",
    "alternative_titles": "",
    "apikey": "d4df59fe284dcd60308b0a5b68a3ac6",
    "backlog": 1,
    "digital_release_date": null,
    "downloadid": "230",
    "finished_date": null,
    "finished_file": null,
    "finished_score": 660,
    "guid": "https//api.nzb.su/getnzb/966f90a42d59298a0e933a86f23ba925.nzb&i=71002&r=xxxxxx",
    "imdbid": "tt1182345",
    "mode": "failed",
    "movie_file": "",
    "path": "D:\\cache nzbget\\intermediate\\Moon.2009.1080p.Bluray.x264.DTS-WiKi.#231",
    "plot": "With only three weeks left in his three year contract, Sam Bell is getting anxious to finally return to Earth. He is the only occupant of a Moon-based manufacturing facility along with his computer and assistant, GERTY. The long period of time alone however has resulted in him talking to himself for the most part, or to his plants. Direct communication with Earth is not possible due to a long-standing communication malfunction but he does get an occasional message from his wife Tess. When he has an accident however, he wakens to find that he is not alone. He also comes to realize that his world is not what he thought it was.",
    "poster": "images/poster/tt1182345.jpg",
    "predb": "found",
    "quality": "Default",
    "rated": null,
    "release_date": "2009-06-12",
    "resolution": "BluRay-1080P",
    "score": "7.6",
    "status": "Snatched",
    "title": "Moon",
    "tmdbid": "17431",
    "url": "https://www.themoviedb.org/movie/17431",
    "year": "2009"
  },
  "status": "finished",
  "tasks": {
    "autograb": {
      "enabled": true,
      "response": false
    },
    "cleanup": {
      "enabled": true,
      "path": "D:\\cache nzbget\\intermediate\\Moon.2009.1080p.Bluray.x264.DTS-WiKi.#231",
      "response": false
    },
    "guid": {
      "update_MARKEDRESULTS": true,
      "update_SEARCHRESULTS": false,
      "url": "https//api.nzb.su/getnzb/966f90a42d59298a0e933a86f23ba925.nzb&i=71002&r=xxxxx"
    },
    "update_movie_status": true
  }
}

I am using sabnzbd and transmission. Watcher snatches the file just fine, but when it finished nothing happens. The status stays set to snatched and the file never gets renamed or moved. Both of my downloaders are on the same both that Watcher3 is on.

A side not when I mark a file as bad in watcher it doesn't cancel the download.

Hello,

The connector to qbittorrent seems broken.

qbittorrent is running on the same server as Watcher on port 8080 with login password.

I can login to the Http interface of qbittorrent with password with local or external ip. I can go to the source of a magnet by Watcher, enter it in QB HTTP interface and the download will start

When I try to setup the downloader on qbittorrent with either localhost, 127.0.0.1, 0.0.0.0, lan IP or external ip i only get a red warning box with no message provided.

Qbittorrent does not log any connection attempt or magnets.

The config.cfg takes parameters into account.

When i click on a torrent it says 'magnet send to qbittorrent' in green,

Here some relevant log

ERROR 2017-05-12 12:03:28,875 core.downloaders.qbittorrent._get_download_dir: QBittorrent unable to get download dir. Traceback (most recent call last): File "/home/alex/Medias/.apps/Watcher3/core/downloaders/qbittorrent.py", line 87, in _get_download_dir response = json.loads(Url.open(url, headers=headers).text) File "/usr/lib/python3.4/json/__init__.py", line 318, in loads return _default_decoder.decode(s) File "/usr/lib/python3.4/json/decoder.py", line 343, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode raise ValueError(errmsg("Expecting value", s, err.value)) from None ValueError: Expecting value: line 1 column 1 (char 0) INFO 2017-05-12 12:03:28,910 core.snatcher.snatch_torrent: Successfully sent Blood.Father.2016.BluRay.1080p.x264.DTS-HDChina[PRiME] to QBittorrent.

Hello,
I installed Watcher3 on windows 10 using GIT.
I can start it, configure it. But I have this error message :
"DEBUG 2017-04-10 14:16:02,237 core.notification.add: {'type': 'warning', 'title': 'Error Checking for Updates', 'body': ['[WinError 2] Le fichier spécifié est introuvable'], 'params': '{closeButton: true, timeOut: 0, extendedTimeOut: 0}', 'closeButton': 'true'}
DEBUG 2017-04-10 14:16:02,237 core.notification.add: Creating new notification:
INFO 2017-04-10 14:16:02,236 core.version.update_check: Checking git for a new version.
ERROR 2017-04-10 14:16:02,236 core.version.check_git_available: Could not execute git: ['[WinError 2] Le fichier spécifié est introuvable']
FileNotFoundError: [WinError 2] Le fichier spécifié est introuvable
startupinfo)
File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 990, in _execute_child
restore_signals, start_new_session)
File "C:\Program Files (x86)\Python36-32\lib\subprocess.py", line 707, in init
creationflags=CREATE_NO_WINDOW
File "c:\watcher3\core\version.py", line 63, in runner
Traceback (most recent call last):
ERROR 2017-04-10 14:16:02,185 core.version.runner: Subprocess:"

I don't why it can't find GIT. Is it possible to specify where git is installed ?

Did an update today.

https://github.com/nosmokingbandit/watcher3/compare/66cfaffc3bef36185717817ce37dbcc7b0d6e423...f413f3989d0871a4f8f86d54194deb4b0c7892fb

I received some PP errors in nzbget 18.1.

nzbget: Traceback (most recent call last): nzbget: File "<frozen importlib._bootstrap>", line 2218, in _find_and_load_unlocked nzbget: AttributeError: 'module' object has no attribute '__path__' nzbget: During handling of the above exception, another exception occurred: nzbget: Traceback (most recent call last): nzbget: File "/home/nzbget/scripts/nzbget.py", line 37, in <module> nzbget: import urllib.parse.urlencode as urlencode nzbget: ImportError: No module named 'urllib.parse.urlencode'; 'urllib.parse' is not a package Post-process-script scripts/nzbget.py for XXXXXXXXX failed (terminated with unknown status)

pip search urlparse output.
pip search urlparse giturlparse.py (0.0.5) - A Git URL parsing module (supports parsing and rewriting) murl (0.5.1) - murl is a tiny wrapper for the Python module urlparse. uritools (2.0.0) - RFC 3986 compliant, Unicode-aware, scheme-agnostic replacement for urlparse urlparse2 (1.1.1) - urlparse for humans. urlparse3 (1.1) - urlparse3 is simple and powerful url parsing tool urlparse4 (0.1.3) - Performance-focused replacement for Python's urlparse module

Installed python 2.7.9 and python 3.4 on Debian Jessie.

Hey,
since fd77a42 I got a wired error...

Traceback (most recent call last):
File "watcher.py", line 17, in
import core
File "/root/watcher/core/init.py", line 31, in
NO_VERIFY = ssl._create_unverified_context()
AttributeError: 'module' object has no attribute '_create_unverified_context'

Could you please fix this asap!

WARNING:core.snatcher:Unable to automatically grab tt1856101, no results.
INFO:core.sqldb:Retrieving list of user's movies.
Myip - Myuser [11/May/2017:09:47:02] "GET /status/ HTTP/1.1" 200 20909 "http://Mysite.ddns.us:9090/add_movie/" "Mozilla/5.0 (Windows NT 10.0; Win64; x6 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.96 Safari/537.36"
Myip - Myuser [11/May/2017:09:47:02] "GET /static/images/posters/tt1856 101.jpg HTTP/1.1" 200 16009 "http://Mysite.ddns.us:9090/status/" "Mozilla/5.0 (W indows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3 029.96 Safari/537.36"
INFO:core.rss.predb:Checking predb.me for new available releases.
INFO:core.sqldb:Retrieving list of user's movies.
INFO:core.rss.predb:Checking predb.me for new available releases for A Dog's Pur pose.
Exception in thread Thread-12:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 1186, in run
self.function(*self.args, **self.kwargs)
File "/opt/Watcher3/core/cp_plugins/taskscheduler.py", line 122, in task
self.func()
File "/opt/Watcher3/core/searcher.py", line 58, in auto_search_and_grab
self.predb.check_all()
File "/opt/Watcher3/core/rss/predb.py", line 33, in check_all
self.check_one(movie)
File "/opt/Watcher3/core/rss/predb.py", line 61, in check_one
if self.fuzzy_match(rss_titles, test):
File "/opt/Watcher3/core/rss/predb.py", line 118, in fuzzy_match
match = fuzz.partial_ratio(item, test)
File "/opt/Watcher3/lib/fuzzywuzzy/utils.py", line 23, in decorator
return func(*args, **kwargs)
File "/opt/Watcher3/lib/fuzzywuzzy/utils.py", line 32, in decorator
return func(*args, **kwargs)
File "/opt/Watcher3/lib/fuzzywuzzy/fuzz.py", line 69, in partial_ratio
blocks = m.get_matching_blocks()
File "/opt/Watcher3/lib/fuzzywuzzy/StringMatcher.py", line 59, in get_matching _blocks
self._str1, self._str2)
TypeError: inverse expected a list of edit operations

I'm trying to run on my Ubuntu 14.04. I cloned the project with git and try to execute like:

#: python3 watcher.py

but I'm the following exception

# python3 watcher.py 
/opt/watcher/lib/fuzzywuzzy/fuzz.py:35: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Opening SQL DB.
Traceback (most recent call last):
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 337, in dbapi
    from sqlite3 import dbapi2 as sqlite  # try 2.5+ stdlib name.
  File "/usr/lib/python3.4/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/lib/python3.4/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: /usr/lib/python3.4/lib-dynload/_sqlite3.cpython-34m-arm-linux-gnueabihf.so: undefined symbol: sqlite3_transfer_bindings

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/watcher/core/sqldb.py", line 23, in __init__
    self.engine = create_engine(DB_NAME, echo=False, connect_args={'timeout': 30})
  File "/opt/watcher/lib/sqlalchemy/engine/__init__.py", line 387, in create_engine
    return strategy.create(*args, **kwargs)
  File "/opt/watcher/lib/sqlalchemy/engine/strategies.py", line 80, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 339, in dbapi
    raise e
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 334, in dbapi
    from pysqlite2 import dbapi2 as sqlite
ImportError: No module named 'pysqlite2'
Traceback (most recent call last):
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 337, in dbapi
    from sqlite3 import dbapi2 as sqlite  # try 2.5+ stdlib name.
  File "/usr/lib/python3.4/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/lib/python3.4/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: /usr/lib/python3.4/lib-dynload/_sqlite3.cpython-34m-arm-linux-gnueabihf.so: undefined symbol: sqlite3_transfer_bindings

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "watcher.py", line 22, in <module>
    from core import api, config, postprocessing, scheduler, sqldb
  File "/opt/watcher/core/api.py", line 5, in <module>
    from core import ajax, sqldb
  File "/opt/watcher/core/ajax.py", line 9, in <module>
    from core import config, library, plugins, searchresults, searcher, snatcher, sqldb, version
  File "/opt/watcher/core/config.py", line 6, in <module>
    from core import scheduler
  File "/opt/watcher/core/scheduler.py", line 8, in <module>
    from core import searcher, version
  File "/opt/watcher/core/searcher.py", line 5, in <module>
    from core import library, searchresults, snatcher, sqldb, proxy
  File "/opt/watcher/core/library.py", line 20, in <module>
    sql = sqldb.SQL()
  File "/opt/watcher/core/sqldb.py", line 23, in __init__
    self.engine = create_engine(DB_NAME, echo=False, connect_args={'timeout': 30})
  File "/opt/watcher/lib/sqlalchemy/engine/__init__.py", line 387, in create_engine
    return strategy.create(*args, **kwargs)
  File "/opt/watcher/lib/sqlalchemy/engine/strategies.py", line 80, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 339, in dbapi
    raise e
  File "/opt/watcher/lib/sqlalchemy/dialects/sqlite/pysqlite.py", line 334, in dbapi
    from pysqlite2 import dbapi2 as sqlite
ImportError: No module named 'pysqlite2'

Downloaded movies remain as Snatched, no updates after the movie has finished downloading. I've checked watcher logs and did not find any other changes after the following:
INFO 2017-05-19 17:09:05,324 core.sqldb.update: Updating status to Snatched for col <movie> in MOVIES. INFO 2017-05-19 17:09:05,324 core.library.movie_status: Setting MOVIES <movie> status to Snatched.
I checked recent Sabnzbd logs as well but could not find any error messages related to the download.

Sabnzbd version: 2.0.0 [55c4bef]

TypeError: object of type 'bool' has no len() length = len(movie_data) + len(corrected_movies) File "/opt/watcher3/app/core/ajax.py", line 699, in submit_import for chunk in body: File "/opt/watcher3/app/lib/cherrypy/lib/encoding.py", line 100, in encoder return next(self.iter_response) File "/opt/watcher3/app/lib/cherrypy/_cpwsgi.py", line 286, in __next__ return func(*args, **kwargs) File "/opt/watcher3/app/lib/cherrypy/_cpwsgi.py", line 189, in trap

I can import individual files but when i add the full lot of movies (500~) it throws a 500 internal error.

Will send an email with full logs.

I see that multiple movies appear 2-4 times in my watcher status page... Is this a known bug?

i have tried to download several movies and each time when they complete, they are not moved to my movie folder. Here is the last log entry in qbittorrent for it:

(N) 2017-04-22T23:59:59 - Torrent: A.Movie's.Title.2017, running external program, command: C:\Program Files\SABnzbd\scripts\qbittorrent.py"A.Movie's.Title.2017" "J:\Downloads\Watcher" "J:\Downloads\Watcher\A.Movie's.Title.2017" "e40b2dfdf8e080db7c96c43c40ad69b376243efe"

there are no log entries in the watcher logs whatsoever for this as well.

*Edited for content by @nosmokingbandit

I'm not sure this is the fault of Watcher, and could just be a quirk of rtorrent. The process is different for magnets vs torrents, but the end result is the same. This only applies to snatches using rtorrent HTTP - not scgi.

For magnets:
Watcher seems to be sending the GUID.meta to the client successfully, and the submission does seem to contain the label correctly. You can briefly see the GUID download in the UI with a status of "Queued", and it does have a Watcher label on it. The problem then happens when rtorrent does its magic to convert that GUID.meta into a proper titled and active download. At that point when the file starts downloading, the label disappears.

For torrents:
Watcher sends the .torrent to the client, and it goes straight into active download. It obviously doesn't need to convert the .meta, so that step gets skipped. Unfortunately, even though the process is more straight forward, the download still has no Watcher label on it for some reason.

As a result, regardless of whether its a magnet or torrent source, once the download is complete with no label on it the post processing fails.

There isn't anything significant in the logs unfortunately. Everything looks successful as far as Watcher is concerned.

Like I said, this may be inherent quirkiness of rtorrent rather than Watcher, but it does limit the usability of rtorrent / HTTP as a download mechanism.

rtorrent / libtorrent : 0.9.6 / 0.13.6

Restart button, to restart Watcher

Found restart and shutdown under the server option in settings.

Noticed nzbget post-processing wasn't working anymore after updating to latest versions of everything. See output below when trying to run test.py from the command prompt.

C:\Watcher3\post scripts>c:\Python36-32\python.exe test.py
Traceback (most recent call last):
File "test.py", line 23, in
import urllib.parse.urlencode as urlencode
ModuleNotFoundError: No module named 'urllib.parse.urlencode'; 'urllib.parse' is not a package

C:\Watcher3\post scripts>c:\Python36-32\python.exe test2.py
Traceback (most recent call last):
File "test2.py", line 23, in
import urllib.parse.urlencode as urlencode
ModuleNotFoundError: No module named 'urllib.parse.urlencode'; 'urllib.parse' is not a package

C:\Watcher3\post scripts>

Posted to separate issue from #27

File "/root/watcher/core/postprocessing.py", line 223, in get_movie_info
data.update(self.metadata.get_metadata(data['movie_file']))

Happens when post-processing movie. Odd since the line directly before this method is called assigns that key to the the data dict.

I'm working on it.

Requesting that there are some other import options as Google will ban if too many requests for opening the file headers (It thinks its a full file download)

1. Import from Plex
2. Import from Radarr
3. Import from Couchpotato

Cheers

I'm using rTorrent SCGI as a downloader but sending torrents and magnets to it from Watcher doesn't seem to actually work. I dont think its a problem with my rTorrent config because im using it successfully with other searchers (Medusa and Mylar).

Logs as follows:

INFO 2017-04-17 14:53:04,101 core.snatcher.snatch_torrent: Sending magnet to rTorrent.
ERROR 2017-04-17 14:53:04,102 cherrypy.error.1973115664.error: [17/Apr/2017:14:53:04] HTTP
Traceback (most recent call last):
  File "/home/watcher/lib/cherrypy/_cprequest.py", line 670, in respond
    response.body = self.handler()
  File "/home/watcher/lib/cherrypy/lib/encoding.py", line 220, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/home/watcher/lib/cherrypy/_cpdispatch.py", line 60, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/watcher/core/ajax.py", line 307, in manual_download
    return json.dumps(self.snatcher.snatch(data))
  File "/home/watcher/core/snatcher.py", line 129, in snatch
    response = self.snatch_torrent(data)
  File "/home/watcher/core/snatcher.py", line 277, in snatch_torrent
    response = rtorrent.rTorrentSCGI.add_torrent(data)
  File "/home/watcher/core/downloaders/rtorrent.py", line 55, in add_torrent
    connected = rTorrentSCGI.test_connection(conf)
  File "/home/watcher/core/downloaders/rtorrent.py", line 28, in test_connection
    client = rtorrent.SCGIServerProxy(url)
  File "/home/watcher/lib/rtorrent.py", line 142, in __init__
    type, uri = urllib.splittype(uri)
AttributeError: module 'urllib' has no attribute 'splittype'
INFO 2017-04-17 14:53:04,184 cherrypy.error.1973115664.error: [17/Apr/2017:14:53:04] HTTP
Request Headers:
  ACCEPT: */*
  REFERER: http://obfuscated:9090/watcher/
  Remote-Addr: 192.168.0.1
  ORIGIN: http://obfuscated:9090
  X-REQUESTED-WITH: XMLHttpRequest
  ACCEPT-ENCODING: gzip, deflate
  COOKIE: much obfuscated data
  Content-Length: 107
  USER-AGENT: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
  Content-Type: application/x-www-form-urlencoded; charset=UTF-8
  ACCEPT-LANGUAGE: en-US,en;q=0.8
  HOST: obfuscated:9090
  CONNECTION: keep-alive

Clicking the "Test Connection" button times out with the same error in the logs.

rTorrent/libtorrent versions are 0.9.6/0.13.6 which is running on the same host as Watcher, which is master commit 31e50c9 all on Ubuntu 16.04.2 armhf on a Pi2.

EDIT: Package versions are:

ii python3 3.5.1-3 armhf
ii python3-urllib3 1.13.1-2ubuntu0.16.04.1 all

Let me know if theres any other info you need.

Cheers :)

It may be a unique selling point for this tool to support OneClickHoster as well as torrent and nzb. At least I've found so far no other tool (Sonarr, Radarr, CouchPotato, Sickbeard...) which can offer this. Torrent and usenet are quite nice, but Torrent is in many terms not the best solution for everyone and usenet is quite expensive and very tricky to reach a good nzb-indexer (especially for foreign content)

As far as I can estimate, implementation shouldn't be a really big deal. There are already some nice Tools (jdownloader, pyload9, which are handling the decryption of the hosting sites. So all we have to do is scraping the content-sites and send links to the downloader's via API POST Requests.

The search procedure uses imdbid on piratebay to search. That reduces the number of matches significantly since a lot of torrents don't include imdbid.
I would suggest to perform a movie name search like couchpotato does.

Hi,

i have a big preferred word list to manage not taking bad release groups. It will be nice if watcher look at multiple words and stack the score by every hit.
Now it will stop finding by 1 and raise the score only by 10.
But to have a bigger score impact, it will be nice if watcher stack the score by finding multiple matches :)

Failed while doing manual search

Error:

xml.etree.ElementTree.ParseError: mismatched tag: line 27, column 2
  File "<string>", line None
    parser.feed(text)
  File "/usr/lib/python3.4/xml/etree/ElementTree.py", line 1325, in XML
    tree = ET.fromstring(xml)
  File "/opt/Watcher3/core/providers/torrent.py", line 434, in parse
    return ExtraTorrent.parse(response, imdbid)
  File "/opt/Watcher3/core/providers/torrent.py", line 397, in search
Traceback (most recent call last):
ERROR 2017-05-31 10:22:24,535 core.providers.torrent.search: ExtraTorrent search failed.

Follow up from this

Batch file executes the python script now, but still outputs an error.

[WARNING ] 17:55:40 core:132 [execute] command 'C:\Watcher3\scripts\deluge.bat' failed with exit code 1
[WARNING ] 17:55:40 core:136 [execute] stderr: Traceback (most recent call last):

  File "C:\Watcher3\scripts\\deluge.py", line 49, in <module>

    response = json.loads(urllib.request.urlopen(request, timeout=600).read())

  File "C:\Python36\lib\urllib\request.py", line 223, in urlopen

    return opener.open(url, data, timeout)

  File "C:\Python36\lib\urllib\request.py", line 524, in open

    req = meth(req)

  File "C:\Python36\lib\urllib\request.py", line 1248, in do_request_

    raise TypeError(msg)

TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

Request to add option to scan the downloads directory for occasions where post processing has failed.

Ubuntu 14.04.1
Python 3.4.3

Fresh git pull. Unable to start.

user@ubuntu /o/watcher> python3 /opt/watcher/watcher.py --daemon
Traceback (most recent call last):
  File "/opt/watcher/watcher.py", line 13, in <module>
    from core import api, config, postprocessing, scheduler, sqldb
  File "/opt/watcher/core/api.py", line 6, in <module>
    from core import ajax, sqldb, poster
  File "/opt/watcher/core/ajax.py", line 9, in <module>
    from core import config, library, plugins, poster, scoreresults, searcher, snatcher, sqldb, updatestatus, version
  File "/opt/watcher/core/searcher.py", line 5, in <module>
    from core import scoreresults, snatcher, sqldb, updatestatus, proxy
  File "/opt/watcher/core/snatcher.py", line 5, in <module>
    from core import plugins, sqldb, updatestatus
  File "/opt/watcher/core/updatestatus.py", line 3, in <module>
    from core import sqldb, library, scoreresults, searcher
ImportError: cannot import name 'searcher'

searcher.py is in core/

user@ubuntu /o/watcher> ls core
ajax.py          conf_api.ini     conf_postprocessing.ini  __init__.py      plugins.py         __pycache__/     snatcher.py
api.py           conf_app.ini     cp_plugins/              library.py       poster.py          rss/             sqldb.py
app.py           conf_auth.ini    downloaders/             log.py           postprocessing.py  scheduler.py     updatestatus.py
auth.py          conf_global.ini  favicon.ico              movieinfo.py     providers/         scoreresults.py  version.py
base_config.cfg  config.py        helpers.py               notification.py  proxy.py           searcher.py

If you got a huge list in watcher you need some time to load the whole list. It would help if there is a default view of 100 lines and multiple sites and maybe an option to view all lines of the list.

Thanks :)

If a movie has single quotes in its description, the quotes are not escaped, which leads to invalid JSON (everything after the quotes is added as element attributes) being saved and later sent to the server.

Starting up a fresh download of version 9a36a0b on windows with python version
3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)]
using
py -3 watcher.py
results in following crash:

<.....>\Watcher3-master\lib\fuzzywuzzy\fuzz.py:35: UserWarning: Usin
g slow pure-python SequenceMatcher. Install python-Levenshtein to remove this wa
rning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levensht
ein to remove this warning')
Config file found, merging any new options.
Database found.
Traceback (most recent call last):
File "watcher.py", line 123, in
root = cherrypy.tree.mount(App(),
File "<.....>\Watcher3-master\core\app.py", line 35, in init
scheduler.AutoUpdateCheck.update_check()
File "<.....>\Watcher3-master\core\scheduler.py", line 75, in upda
te_check
ver = version.Version()
File "<.....>\Watcher3-master\core\version.py", line 28, in __init
__
self.manager = ZipUpdater()
File "<.....>\Watcher3-master\core\version.py", line 254, in ini
t
core.CURRENT_HASH = self.current_hash.decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'

Log above is from a repeated startup where a default config file was found that was created by a previous startup crashing with the same error.

Is 3.4 python just to old that is has no "decode" function on every string object? I could not find any other version requirements other than "3"

(I apologize first time submitting an issue so I hope I have everything you need.)
Here is my post processing log:

INFO 2017-04-07 10:24:53,620 core.postprocessing.POST: #################################
INFO 2017-04-07 10:24:53,620 core.postprocessing.POST: Post-processing request received.
INFO 2017-04-07 10:24:53,620 core.postprocessing.POST: #################################
INFO 2017-04-07 10:24:53,621 core.postprocessing.get_movie_file: Finding movie file.
INFO 2017-04-07 10:24:53,621 core.postprocessing.get_movie_file: Post-processing file /share/Download/nzbget/dst/Watcher//.mkv.
INFO 2017-04-07 10:24:53,621 core.postprocessing.POST: Parsing release name for information.
INFO 2017-04-07 10:24:53,621 core.library.parse_filename: Parsing /share/Download/nzbget/dst/Watcher//.mkv for movie information.
INFO 2017-04-07 10:24:53,622 core.library.parse_filename: Found {'episode': 94, 'container': 'mkv', 'title': '', 'episodeName': ''} in filename.
INFO 2017-04-07 10:24:53,623 core.postprocessing.POST: Gathering release information.
INFO 2017-04-07 10:24:53,623 core.postprocessing.get_movie_info: Searching local database for guid.
INFO 2017-04-07 10:24:53,623 core.sqldb.get_single_search_result: Retrieving search result details for http://localhost:5075/getnzb?apikey=.
INFO 2017-04-07 10:24:53,625 core.postprocessing.get_movie_info: Searchresult found by guid.
INFO 2017-04-07 10:24:53,625 core.postprocessing.get_movie_info: Searching local database by imdbid.
INFO 2017-04-07 10:24:53,625 core.sqldb.get_movie_details: Retrieving details for tt.
INFO 2017-04-07 10:24:53,627 core.postprocessing.get_movie_info: Movie data found locally by imdbid.
INFO 2017-04-07 10:24:53,628 core.postprocessing.POST: Post-processing as Complete.
INFO 2017-04-07 10:24:53,628 core.postprocessing.complete: Marking guid as Finished.
INFO 2017-04-07 10:24:53,628 core.library.searchresults: Marking guid http_//localhost_5075/getnzb_apikey= as Finished.
INFO 2017-04-07 10:24:53,630 core.library.searchresults: Guid http_//localhost_5075/getnzb_apikey= not found in SEARCHRESULTS, attempting to create entry.
ERROR 2017-04-07 10:24:53,630 cherrypy.error.139854216156888.error: [07/Apr/2017:10:24:53] HTTP
Traceback (most recent call last):
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/lib/cherrypy/_cprequest.py", line 670, in respond
response.body = self.handler()
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/lib/cherrypy/lib/encoding.py", line 220, in call
self.body = self.oldhandler(*args, **kwargs)
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/lib/cherrypy/_cpdispatch.py", line 60, in call
return self.callable(*self.args, **self.kwargs)
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/core/postprocessing.py", line 102, in POST
response = self.complete(data)
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/core/postprocessing.py", line 364, in complete
if self.update.searchresults(data['guid'], 'Finished', movie_info=data):
File "/share/CACHEDEV1_DATA/.qpkg/watcher3/core/library.py", line 639, in searchresults
search_result = searchresults.generate_simulacrum(movie_info)
TypeError: 'staticmethod' object is not callable
INFO 2017-04-07 10:24:53,631 cherrypy.error.139854216156888.error: [07/Apr/2017:10:24:53] HTTP
Request Headers:
Remote-Addr: 127.0.0.1
ACCEPT-ENCODING: identity
HOST: localhost:9090
CONNECTION: close
USER-AGENT: Mozilla/5.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 307

Will be very nice stuff if its possible to have a auto add function based on list from trakt.tv (like radarr has)

• Trending
• Popular
• Watched
• Collected
• Anticipated
• Box Office

Will be nice if you can config some settings likes: Need a higher score then 72 and not more results then 10 and how mutch time i will check again.

Search is just take in consideration the first page.
If the the results are paginated through several pages, only the first one is being considered.

It will be nice if we show how the movie is added.

Status can be: API, added manual, Automatic added from list or automatic added from trakt.tv

The field for the api key is gone after last update :(

If a release name has DTS in it will get flagged as having TS in it.

INFO 2017-05-14 00:56:25,389 cherrypy.error.140452261263568.error: [14/May/2017:00:56:25] HTTP
TypeError: string indices must be integers
tmdbid = data['id']
File "/opt/Watcher3/core/library.py", line 630, in add_movie
r = self.manage.add_movie(data, full_metadata=full_metadata)
File "/opt/Watcher3/core/ajax.py", line 103, in add_wanted_movie
return self.callable(*self.args, **self.kwargs)
File "/opt/Watcher3/lib/cherrypy/_cpdispatch.py", line 60, in call
self.body = self.oldhandler(*args, **kwargs)
File "/opt/Watcher3/lib/cherrypy/lib/encoding.py", line 220, in call
response.body = self.handler()
File "/opt/Watcher3/lib/cherrypy/_cprequest.py", line 670, in respond
Traceback (most recent call last):
ERROR 2017-05-14 00:56:25,385 cherrypy.error.140452261263568.error: [14/May/2017:00:56:25] HTTP
INFO 2017-05-14 00:56:21,149 core.movieinfo._search_title: Searching TMDB https://api.themoviedb.org/3/search/movie?

Right now we have the workflow:
Wanted>Found>Snatched>Finished

I like to have one more status: "Waiting" or something like "waiting to be released."
People can set a delay, it is nice to see if watching is still in the release state instead of showing for every requests "wanted".
Beceause, if you have something like this: its more clear what watcher is doing.

What about some statistics we all love stats...
You have ZZZ movies in your library ( AA with status wanted, BB with status found ...)

Maybe it's possible to build something like this?

Hi,

I keep getting this error from sab when it tries to download a movie sent to it by watcher:

Traceback (most recent call last):
File "C:\Program Files\SABnzbd\scripts\sabnzbd.py", line 22, in
import urllib.request
ImportError: No module named request

Hi,

You closed the other issue so I repost it here.

Just cloned this new repo and got another error.
Debian Jessie - Python 3.4.2

/usr/bin/python3 /opt/Watcher3/watcher.py --port 2932

/opt/Watcher3/lib/fuzzywuzzy/fuzz.py:35: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')

Traceback (most recent call last):
  File "/opt/Watcher3/watcher.py", line 30, in <module>
    ssl._create_default_https_context = ssl._create_unverified_context
AttributeError: 'module' object has no attribute '_create_unverified_context'

Hello, I just installed clean from GIT and I'm getting the following error when I try to start Watcher3.

pi@raspberrypi:/opt $ sudo python3 Watcher3/watcher.py
/opt/Watcher3/lib/fuzzywuzzy/fuzz.py:35: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Traceback (most recent call last):
File "Watcher3/watcher.py", line 22, in
from core import api, config, postprocessing, scheduler, sqldb
File "/opt/Watcher3/core/api.py", line 5, in
from core import ajax, sqldb
File "/opt/Watcher3/core/ajax.py", line 9, in
from core import config, library, plugins, searchresults, searcher, snatcher, sqldb, version
File "/opt/Watcher3/core/config.py", line 6, in
from core import scheduler
File "/opt/Watcher3/core/scheduler.py", line 9, in
from core.rss import imdb, popularmovies
File "/opt/Watcher3/core/rss/imdb.py", line 2, in
from core import ajax, sqldb
ImportError: cannot import name 'ajax'

Unsure what is going on as I know the file is there in core...

I'll be the first to admit, I really don't know my way with Python. I was hoping that there would be a way to (maybe with a plugin) to make an API call to a Plex Server after Post-Processing has finished... to have it update the Movies Library. (Instead of having to schedule an Library update every few hours)

I have tried adding my "best-effort" to the end of the sabnzbd.py post-processing script... but it failed miserably after moving to Python3.

Definitely not the highest priority, but it would be nice to see down the road. Keep up the great work, this project is amazing!

Maybe I don't see this option but it would help me a lot if there would be an option where I could mark all snateched/wanted/found and force a search or auto snatch. I mean with around 20k of movies watcher is a bit slow so manual searches took more than 5 min per movie :(

It's okay with Usenet providers but not at all with Bittorrent ones, using Jackett.

Here is what Jacketts log showing:
Found 0 releases from T411 for: zootopia2016
Does this mean it searches for "movieyear" and not "movie year"?

Randomly started getting error 500, even after reboot etc. Not really sure how to get things back to normal. Any help is appreciated.

500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
File "/home/ownz0rjoo/watcher3/lib/cherrypy/_cprequest.py", line 670, in respond
response.body = self.handler()
File "/home/ownz0rjoo/watcher3/lib/cherrypy/lib/encoding.py", line 220, in call
self.body = self.oldhandler(*args, **kwargs)
File "/home/ownz0rjoo/watcher3/lib/cherrypy/_cpdispatch.py", line 60, in call
return self.callable(*self.args, **self.kwargs)
File "/home/ownz0rjoo/watcher3/templates/status.py", line 49, in default
self.movie_list()
File "/home/ownz0rjoo/watcher3/templates/status.py", line 97, in movie_list
span(data['year'], cls='year')
File "/home/ownz0rjoo/watcher3/lib/dominate/tags.py", line 63, in init
super(html_tag, self).init(*args, **kwargs)
File "/home/ownz0rjoo/watcher3/lib/dominate/dom_tag.py", line 93, in init
self.add(*args)
File "/home/ownz0rjoo/watcher3/lib/dominate/dom_tag.py", line 204, in add
raise ValueError('%r not a tag or string.' % obj)
ValueError: None not a tag or string.
Powered by CherryPy unknown

SABnzbd: Version: 2.0.0RC2 [afd1b19]
Python 3.5.2
--proxmox container ubuntu 16.04

I updated the script to the new watcher3 version and this is the error i get in sabnzbd

Traceback (most recent call last):
  File "/home/sabnzbd/scripts/watcher.py", line 47, in <module>
    slots = json.loads(response)['history']['slots']
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

INFO 2017-04-25 12:32:17,794 core.postprocessing.POST: #################################
}
  }
    "update_movie_status": false
    },
      "response": true
      "enabled": true,
    "renamer": {
    },
      "response": true
      "enabled": true,
    "mover": {
    },
      "response": false
      "enabled": true,
    "cleanup": {
  "tasks": {
  "status": "finished",
  },
    "year": ""
    "videocodec": "MPEG4",
    "url": "https://www.themoviedb.org/movie/<id>",
    "tmdbid": "imdbid",
    "title": "b323fff12b",
    "status": "Snatched",
    "size": "",
    "season": 8,
    "score": "6.4",
    "resolution": "",
    "releasegroup": "",
    "release_date": "<release_date>",
    "rated": null,
    "quality": "Default",
    "poster": "images/poster/tt____IMDB.jpg",
    "plot": "heart warming plot",
    "path": "/mnt/media/Downloads/completed/Movies/sweet movie/b323fff12b8e4511818f3017700e9391.mkv",
    "original_file": "/mnt/media/Downloads/completed/Movies/sweet movie/b323fff12b8e4511818f3017700e9391.mkv",
    "movie_file": "/mnt/media/Downloads/completed/Movies/sweet movie/b323fff12b.mkv",
    "mode": "complete",
    "imdbid": "",
    "guid": "http//mediaserver.localdomain5075/nzbhydra/getnzbapikey=<api>&searchresultid=<search id>",
    "finished_score": 620,
    "finished_file": "/mnt/media/Videos/Movies/b323fff12b ()/b323fff12b.mkv",
    "finished_date": "2017-04-25",
    "downloadid": "34",
    "digital_release_date": null,
    "container": "mkv",
    "audiocodec": "DTS",
    "apikey": "<api>",
    "alternative_titles": "",
    "added_date": "2017-04-25",
  "data": {
INFO 2017-04-25 12:32:17,793 core.postprocessing.POST: {
INFO 2017-04-25 12:32:17,793 core.postprocessing.POST: Post-processing complete.
INFO 2017-04-25 12:32:17,793 core.postprocessing.POST: #################################

It for some reason isn't renaming the file the correct name...

Have just tried to install on a fresh raspberry pi installation
and recieved the following when trying to start
Traceback (most recent call last):
File "watcher.py", line 13, in
from core import api, config, postprocessing, scheduler, sqldb
File "/opt/watcher/core/api.py", line 6, in
from core import ajax, sqldb, poster
File "/opt/watcher/core/ajax.py", line 9, in
from core import config, library, plugins, poster, searchresults, searcher, snatcher, sqldb, version
File "/opt/watcher/core/library.py", line 3, in
from hachoir.parser import createParser
File "/opt/watcher/lib/hachoir/parser/init.py", line 1, in
from hachoir.parser.parser import ValidateError, HachoirParser, Parser # noqa
File "/opt/watcher/lib/hachoir/parser/parser.py", line 142
print("- %s: %s" % (tags["id"], tags["description"]), file=out)
^
SyntaxError: invalid syntax

i am running the latest xbian release

Hey,
I see often not renamed releases as you only try to get the release name from filename.

INFO 2017-04-07 21:27:40,983 core.library.parse_filename: Parsing /PATH/TO/FOLDER/John.Wick.2014.German.AC3D.5.1.DL.720p.BluRay.x264-XDD/JEtzaFunzTOllASwIda_005/fun.mkv for movie information.
INFO 2017-04-07 21:27:40,983 core.library.parse_filename: Found {'container': 'mkv', 'title': 'fun'} in filename.

Now the filename is set to fun but the search string should be john wick instead... Could you fix that?

Today I did a update from the GUI and now Watcher3 won't start

This is the error I am getting:

/opt/Watcher3# python3 /opt/Watcher3/watcher.py  --daemon
Traceback (most recent call last):
  File "/opt/Watcher3/watcher.py", line 22, in <module>
    from core import api, config, postprocessing, scheduler, sqldb
  File "/opt/Watcher3/core/api.py", line 6, in <module>
    from core import ajax, sqldb, poster
  File "/opt/Watcher3/core/ajax.py", line 9, in <module>
    from core import config, library, plugins, poster, searchresults, searcher, snatcher, sqldb, version
  File "/opt/Watcher3/core/config.py", line 6, in <module>
    from core import scheduler
  File "/opt/Watcher3/core/scheduler.py", line 8, in <module>
    from core import searcher, version
  File "/opt/Watcher3/core/searcher.py", line 5, in <module>
    from core import library, searchresults, snatcher, sqldb, proxy
  File "/opt/Watcher3/core/library.py", line 5, in <module>
    from core import sqldb, searchresults, searcher, snatcher, poster, plugins
  File "/opt/Watcher3/core/snatcher.py", line 5, in <module>
    from core import library, plugins, sqldb
ImportError: cannot import name 'library'

After nzbget starts watcher.py it will fail with a 500 http error

ERROR	Mon Apr 24 2017 21:04:34	Post-process-script Watcher.py3 for xxxxxxxxxxxxxx failed (terminated with unknown status)
INFO	Mon Apr 24 2017 21:04:34	Watcher: urllib.error.HTTPError: HTTP Error 500: Internal Server Error
INFO	Mon Apr 24 2017 21:04:34	Watcher: raise HTTPError(req.full_url, code, msg, hdrs, fp)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 650, in http_error_default
INFO	Mon Apr 24 2017 21:04:34	Watcher: result = func(*args)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 504, in _call_chain
INFO	Mon Apr 24 2017 21:04:34	Watcher: return self._call_chain(*args)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 570, in error
INFO	Mon Apr 24 2017 21:04:34	Watcher: 'http', request, response, code, msg, hdrs)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 642, in http_response
INFO	Mon Apr 24 2017 21:04:34	Watcher: response = meth(req, response)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 532, in open
INFO	Mon Apr 24 2017 21:04:34	Watcher: return opener.open(url, data, timeout)
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "C:\Python3\lib\urllib\request.py", line 223, in urlopen
INFO	Mon Apr 24 2017 21:04:34	Watcher: response = json.loads(urlopen(request, timeout=600).read().decode('utf-8'))
INFO	Mon Apr 24 2017 21:04:34	Watcher: File "D:\cache nzbget\scripts\Watcher.py3", line 72, in <module>
INFO	Mon Apr 24 2017 21:04:34	Watcher: Traceback (most recent call last):
INFO	Mon Apr 24 2017 21:04:34	Watcher: Sending xxxxxxxxxxxxxxx to Watcher as Complete.
INFO	Mon Apr 24 2017 21:04:32	Executing post-process-script Watcher.py3 for XXXXXXXXXXXXX

Wacher log:

INFO 2017-04-24 21:04:34,016 cherrypy.error.90438064.error: [24/Apr/2017:21:04:34] HTTP 
KeyError: 'movie_file'
    data.update(self.metadata.get_metadata(data['movie_file']))
  File "C:\watcher\core\postprocessing.py", line 223, in get_movie_info
    data.update(self.get_movie_info(data))
  File "C:\watcher\core\postprocessing.py", line 82, in POST
    return self.callable(*self.args, **self.kwargs)
  File "C:\watcher\lib\cherrypy\_cpdispatch.py", line 60, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "C:\watcher\lib\cherrypy\lib\encoding.py", line 220, in __call__
    response.body = self.handler()
  File "C:\watcher\lib\cherrypy\_cprequest.py", line 670, in respond
Traceback (most recent call last):