sverrirs / ruvsarpur Goto Github PK
View Code? Open in Web Editor NEWPython script to download shows off the Icelandic RÚV website.
Home Page: https://sverrirs.github.io/ruvsarpur/
License: MIT License
Python script to download shows off the Icelandic RÚV website.
Home Page: https://sverrirs.github.io/ruvsarpur/
License: MIT License
Ég er á Linux og með ffmpeg installað. prufaði t.d. "python ruvsarpur.py --voddirect --sid 96060" bara til að prufa og fékk: "PermissionError: [Errno 13] Permission denied: '/home/bjarkih/temp/ruv/bin/ffmpeg.exe" er scriptið að biðja um windows forrit í linux eða er ég að misskilja?
Allt outputið:
Found 7 show(s)
1 of 7: Stiklur (7 af 7) | Estimated: 308 MB
Downloading: |-------------------------| 0.0% StartingTraceback (most recent call last):
File "ruvsarpur.py", line 830, in
runMain()
File "ruvsarpur.py", line 819, in runMain
result = download_m3u8_playlist_using_ffmpeg(ffmpegexec, playlist_data['url'], playlist_data['fragments'], local_filename, display_title, args.keeppartial, args.quality)
File "ruvsarpur.py", line 292, in download_m3u8_playlist_using_ffmpeg
ret = subprocess.Popen(prog_args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True, env=my_env)
File "/usr/lib/python3.7/subprocess.py", line 775, in init
restore_signals, start_new_session)
File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '/home/bjarkih/temp/ruv/bin/ffmpeg.exe'
kemur í hvert skipti núna.. væri mögulegt að laga þetta? :)
þetta virkar ekki eftir að ég uppfærði í Ubuntu server 17.10
fæ alltaf þessa villu
File "/plex/ruvsarpur/src/ruvsarpur.py", line 30, in
from colorama import init, deinit # For colorized output to console windows (platform and shell independent)
ModuleNotFoundError: No module named 'colorama'
PS C:\Users\gisli> python "C:\Users\gisli\Documents\code\ruvsarpur\src\ruvsarpur.py" --sid 34551
Found 4 show(s)
Traceback (most recent call last):
File "C:\Users\gisli\Documents\code\ruvsarpur\src\ruvsarpur.py", line 1565, in
runMain()
File "C:\Users\gisli\Documents\code\ruvsarpur\src\ruvsarpur.py", line 1536, in runMain
result = download_m3u8_playlist_using_ffmpeg(ffmpegexec, playlist_data['url'], playlist_data['fragments'], local_filename, display_title, args.keeppartial, args.quality, args.nometadata, item)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\gisli\Documents\code\ruvsarpur\src\ruvsarpur.py", line 517, in download_m3u8_playlist_using_ffmpeg
if( len(videoInfo['series_sdesc']) > len(series_description)):
^^^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()
Last two weeks I always get an error while using "python ruvsarpur.py --list --refresh" on version ruvsarpur-13.1.0
Unable to retrieve schedule for VOD program 'Krakkafréttir', no episodes will be available for download from this program.
Traceback (most recent call last):
File "ruvsarpur.py", line 986, in getVodSchedule
program_schedule = getVodSeriesSchedule(program['id'], program, imdb_cache, imdb_orignal_titles)
File "ruvsarpur.py", line 1070, in getVodSeriesSchedule
series_title = prog['title'] if isMovie else trimSeasonNumberSuffix(prog['title'])
File "ruvsarpur.py", line 1256, in trimSeasonNumberSuffix
new_series_title = series_title.removesuffix(prefix)
AttributeError: 'str' object has no attribute 'removesuffix'
Then this happen for every show there is.
After it finished he finds only 53 shows.
Is it broken or am I doing something wrong ?
I'm trying to download a particular series. And always on the same particular episode I encounter a non breaking loop.
Edit:
That episode just took an extra long time compared to the others to complete. Sorry
Hello, I've been using your script for a while, usually just set it and forget it, brilliant.
I'm trying out the new version and I'm having difficulties.
Regarding the VOD, is it correct of me to assume that it should be able to fetch any item from the VOD? For example, should the script find all shows from here:
https://www.ruv.is/sjonvarp/spila/utras-i/31642/9dobt8
When I run the following command:
python3 ruvsarpur/src/ruvsarpur.py --list --find "Exit"
I get:
Found 2 show(s)
5168617: Útrás (4 af 8)
30862: Sýnt 2021-03-29 22:20
5168616: Útrás (3 af 8)
30862: Sýnt 2021-03-22 22:20
Regarding the other issue, I run the following command:
python3 ruvsarpur/src/ruvsarpur.py --ffmpeg /usr/bin/ffmpeg --refresh
Results in ( slightly edited )
Downloading VOD schedule | Total: 481 Series in tv
Reading: |=========================| 100.0% Downloading VOD schedule | Total: 242 Series in krakkaruv
Reading: |==-----------------------| 7.0% Traceback (most recent call last):
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/home/user/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
chunked=chunked,
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connection.py", line 182, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f219ac9d2b0>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/user/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/home/user/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 573, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.ruv.is', port=443): Max retries exceeded with url: /gql/?operationName=getEpisode&variables=%7B%22programID%22:30708%7D&extensions=%7B%22persistedQuery%22:%7B%22version%22:1,%22sha256Hash%22:%22f3f9
57a3a577be001eccf93a76cf2ae1b6d10c95e67305c56e4273279115bb93%22%7D%7D (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f219ac9d2b0>: Failed to establish a new connection: [Errno -2] Name or service not known',))
Could it be that RUV is limiting this script or the rate of queries is to fast?
Script is ran on Ubuntu 18.04
Best regards
kemur reglulega
Error: Could not locate VOD download code in VOD data
Fæ upp eftirfarandi ef ég reyni að notast við --category. Prófaði einnig 2,3,4, 5 og 17 og fékk sömu útkomu þar. Einhver hugmynd hvað vandamálið gæti verið?
python ruvsarpur.py --category 1 --list
Nothing found to download
``
PS: annars frábær skrifta. Kærar þakkir fyrir þetta.
python ./src/ruvsarpur.py --ffmpeg /local/bin/ffmpeg -q HD1080 --pid blabla
/local/data/projects/3rd-projects/ruvsarpur/.venv/lib/python3.10/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
The '--imdbfolder' argument is not set, this will impact IMDB matching for non-english content, consider setting this parameter and downloading the 'title.basics.tsv' file from https://www.imdb.com/interfaces/
Downloading VOD schedule | Total: 741 series available
Reading: |-------------------------| 1.9% Unable to retrieve schedule for VOD program 'Allt, alls staðar, alltaf', no episodes will be available for download from this program.
Traceback (most recent call last):
File "/local/data/projects/3rd-projects/ruvsarpur/./src/ruvsarpur.py", line 958, in getVodSchedule
program_schedule = getVodSeriesSchedule(program['id'], program, imdb_cache, imdb_orignal_titles)
File "/local/data/projects/3rd-projects/ruvsarpur/./src/ruvsarpur.py", line 1086, in getVodSeriesSchedule
imdb_cache[str(sid)] = {
TypeError: 'NoneType' object does not support item assignment
Probably because getExistingJsonFile returns None if file doesn't exists.
python3 ruvsarpur.py --refresh --ffmpeg "/usr/bin/ffmpeg"
Unable to retrieve schedule for VOD category 'tv', no episodes will be available for download from this category.
Unable to retrieve schedule for VOD category 'krakkaruv', no episodes will be available for download from this category.
Unable to retrieve schedule for VOD category 'ungruv', no episodes will be available for download from this category.
Nothing found to download
Ég ætlaði að sækja af rúv sarpnum eins og ég geri vikulega og hefur hingað til virkað mjög vel hinsvegar í dag þegar ég gerði --list --refresh kemur hann bara upp með skrá frá klukkan 07:01 í morgun og út mánuðinn bíður ekki upp á fyrri daga og ef ég prufa að gera --find "Brúin kemur bara upp found 5 shows og svo not found on server. prufaði einnig að gera --pid og númmerið sama villa not found on server. hinsvegar virkar fínt að spila ef ég fer beint inná Sarpinn á Rúv.is
Ég er að lenda í því að nýjir þættir sem koma inn á ruv.is, t.d. Blæja, finnast ekki með ruvsarpur. Er einhver bilun eða er ég að misskilja eitthvað?
[ra@stargate src]$ time python3 ./ruvsarpur.py --ffmpeg /usr/bin/ffmpeg --force -q HD720 --pid 5088292
Found 1 show(s)
1 of 1: Útrás (2 af 8) | Estimated: 638 MB
Downloading: |=========================| 100.0% Complete
real 0m0.606s
user 0m0.520s
sys 0m0.055s
638 MB á 0.606 sek er bara asskotigott. Og í kaupbæti er skráin sem er downloaded svo ósýnileg að hún finnst hvergi :)
Ég er hallast að því að þetta sé hreinlega ekkert að downlodast yfir höfuð :)
Ég er að fá eftirfarandi villu þegar ég reyni að uppfæra listann. Þetta virkaði fyrir nokkrum dögum síðan hjá mér.
python ruvsarpur.py --list --refresh
The '--imdbfolder' argument is not set, this will impact IMDB matching for non-english content, consider setting this parameter and downloading the 'title.basics.tsv' file from https://www.imdb.com/interfaces/
python : Traceback (most recent call last):
At line:1 char:1
+ CategoryInfo : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\simplejson\__init__.py", line 514, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\simplejson\decoder.py", line 386, in decode
obj, end = self.raw_decode(s)
^^^^^^^^^^^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\simplejson\decoder.py", line 416, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User\Videos\Sarpurinn\ruvsarpur-12.1.0\src\ruvsarpur.py", line 1545, in
runMain()
File "C:\Users\User\Videos\Sarpurinn\ruvsarpur-12.1.0\src\ruvsarpur.py", line 1349, in runMain
schedule = getVodSchedule(schedule, len(schedule) > 0, imdb_cache, imdb_orignal_titles)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\User\Videos\Sarpurinn\ruvsarpur-12.1.0\src\ruvsarpur.py", line 914, in getVodSchedule
data = r.json()
^^^^^^^^
File "C:\Users\User\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Hi, and again thank you for the effort put into this!
I recently found out my script using your program is returning the following errors:
Error: Could not retrieve episode download url due to parsing error in VOD data, skipping {Series name}
&&
Error: Could not retrieve episode download url, VOD did not return any data, skipping {Series name}
My local version of the repo is up to date. Not aware of any other changes to my system that could cause this.
Any ideas? Maybe RÚV changed something on their end?
Hi, really love this script and I appreciate your work. I am bit confused about the new version which works only with VOD. If I understand it correctly then argument -c --category is obsolete and it can't be no longer used ;( (As explained in issue #12) I can live with that, but I was unable to download anything with the new script verion 7.0.0. I have tried several pids with different quality setting. Eg. pid 5168937. Also with the new version I can´t see some shows. Eg. Salka Valka https://www.ruv.is/sjonvarp/spila/salka-valka/22039/6i5ojh pid 5171888 It was listed in --list from my previous version (4.0.0) and I was able to download it without any issues. With version 7.0.0. it says "Error: Could not retrieve episode download url, VOD did not return any data, skipping" for the same pid
Fæ ekki ffmpeg til að vinna í Ubuntu. Getur þú bent mér á hvernig ég fæ það til virka?
PermissionError: [Errno 13] Permission denied:
Traceback (most recent call last):
File "/ruvsarpur/src/ruvsarpur.py", line 1528, in
runMain()
File "/ruvsarpur/src/ruvsarpur.py", line 1499, in runMain
result = download_m3u8_playlist_using_ffmpeg(ffmpegexec, playlist_data['url'], playlist_data['fragments'], local_filename, display_title, args.keeppartial, args.quality, args.nometadata, item)
File "/ruvsarpur/src/ruvsarpur.py", line 517, in download_m3u8_playlist_using_ffmpeg
if( len(videoInfo['series_sdesc']) > len(series_description)):
TypeError: object of type 'NoneType' has no len()
Nú hef ég ekki notað forritið síðan í byrjun sumar en í gær ætlaði ég að ná í teiknimyndir fyrir krakkana og lendi þá aftur í þessu gamla vandamáli(?)
'Error: Could not retrieve episode download url, unable to download VOD details, skipping Hvolpasveitin VII (1 af 5)'
Er búinn að ná í nýjustu útgáfuna af ruvsarpur en fæ þetta samt sem áður upp. Væri ykkur mjög þakklátur ef það væri til einföld lausn á þessu.
Er staddur í Þýskalandi og nota ExpressVPN.
Hi
When I start ruvsarpur.py it just starts downloading shows. It´s never behaved like this before.
Any ideas?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.