brawl345 / get-dmax-links Goto Github PK

First of all, thanks for your great work.

I just tried to retrieve the video content of this link:
https://dmax.de/sendungen/die-modellbauer-das-miniatur-wunderland/abgerissen/

In the source code is no show id anymore.

<hyoga-player uid="562c9437372439bd" playerselector="hyogaPlayer-562c9437372439bd" videolibrary="videojs" sourcetype="sonic" locale="de" playertype="videoPlayer" assetid="178008" playbacktype="video" autoplay="false" muted="false" id="hyogaManager-562c9437372439bd" endpoint="https://eu1-prod.disco-api.com" realm="dmaxde">

i guess the now use assetid to identify the shows. maybe it's possible to extend the code to use both.

You are only getting the first page of videos for a show. For example "fang-des-lebens-der-gefaehrlichste-job-alaskas" has 122 videos. Since the new api always need pagination, you should check for total amount of pages and get them too.

You could fix it like this (sorry i was to lazy to issue a pull request):
new code:
def get_videos_api_request(showid, token, page):
try:
req = get(SHOW_INFO_URL.format(showid, page), headers={"Authorization": "Bearer " + token})
except Exception as e:
logger.critical("Connection error: {0}".format(str(e)))
return False

if req.status_code != 200:
    logger.error("This show does not exist.")
    return False

data = req.json()
if "errors" in data:
    logger.error("This show does not exist.")
    return False

return data

Replacement for lines 83-96 in dmax.py:
data = get_videos_api_request(showid, token, 1)

if data["meta"]["totalPages"] > 1:
    logger.info("More than 100 videos, need to get more pages")
    for i in range(1, data["meta"]["totalPages"]):
        more_data = get_videos_api_request(showid, token, i+1)
        data["data"].extend(more_data["data"])

When running the script for a show that has more than 30 entries DMAX returns a HTTP error code 429 (Too Many Requests response status code) after 30 requests.
Therefore it is not possible anymore to retrieve the entire list.

2020-09-29 19:22:47,552 - DMAX - INFO - Getting link 29 of 77
2020-09-29 19:22:47,802 - DMAX - INFO - Getting link 30 of 77
2020-09-29 19:22:48,034 - DMAX - INFO - Getting link 31 of 77
2020-09-29 19:22:48,236 - DMAX - ERROR - HTTP error code 429 for video id 63410
2020-09-29 19:22:48,236 - DMAX - INFO - Getting link 32 of 77
2020-09-29 19:22:48,431 - DMAX - ERROR - HTTP error code 429 for video id 63351
2020-09-29 19:22:48,431 - DMAX - INFO - Getting link 33 of 77
2020-09-29 19:22:48,631 - DMAX - ERROR - HTTP error code 429 for video id 63100
2020-09-29 19:22:48,636 - DMAX - INFO - Getting link 34 of 77

After 40 errors the DMAX site returns valid results from request 70 onward. This may vary slightly.

2020-09-29 19:22:55,700 - DMAX - INFO - Getting link 69 of 77
2020-09-29 19:22:55,900 - DMAX - ERROR - HTTP error code 429 for video id 36066
2020-09-29 19:22:55,910 - DMAX - INFO - Getting link 70 of 77
2020-09-29 19:22:56,130 - DMAX - INFO - Getting link 71 of 77
2020-09-29 19:22:56,370 - DMAX - INFO - Getting link 72 of 77
2020-09-29 19:22:56,590 - DMAX - INFO - Getting link 73 of 77

Maybe a configurable "wait" parameter that would delay the script execution could prevent these errors.

Hi,

thank you for the tool.
It appears to me that the websites code has changed - I cant find the showid any longer. Or is it only on my end?

Cheers

Hi,
wäre es möglich den Command auf "yt-dlp" anzupassen? youtube-dl ist selber ja tot

The most channels from Discovery uses the disco-api.
It would be nice to support these sites too. Heres a list:
https://dmax.de/
https://tlc.de/
https://de.hgtv.com/

And maybe rename dmax.py to discovery.py?

Consider the source code of the page https://dmax.de/sendungen/steel-buddies/
The dmax.py script retrieves all shows for
playertype="showPlayer"
showid="6023"

However there are more shows on the same page that are referenced by the following lines
playertype="collectionPlayer"
collectionid="steel-buddies-special-playlist"
Logically these shows won't be in the xlsx file.

Is it possible to include these too?

Cheers,

Windows Defender doesn't like your 64bit .exe:

Trojan:Win32/Wacatac.B!ml

Just tried it with showid 60825 fo e.g. Script nicely output all 10 eposides in csv. I tried the generated urls with yt-dlp and youtube-dl and found, that the url is essential if it works or not:

• Links containing akamaized.net/.../playlist.m3u8 works great
• Links containing akamaized.net/.../hls_fairPlay_ts/.../master.m3u8 doesnt work (seams drm protected, see output below).

Is there any workaround?
Thanks

Sample output of second url:

[generic] master: Downloading webpage
WARNING: [generic] Falling back on generic information extractor.
[generic] master: Downloading m3u8 information
[info] master: Downloading 1 format(s): 9301+audio-aacl-125-English
[hlsnative] Downloading m3u8 manifest
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format

Servus!
Also die neue Version ist ja echt mega schnell, good Job!

Allerdings fehlt in der erstellten Tabelle nun bei mir der Episodennamen im "Command"-Befehl.

youtube-dl...-o "Serie - S11E09.mp4"

Also das hinter der "S11E09" z.B. "Neues freies Land" fehlt.

Danke dir und VG,

Is there a way to update the existing lists with the actual showids?

And: The Parameter ShowId oder show.id isn't on the Sides anymore so it is no longer possible to get it from DMAX, TLC etc. Is this true?

Regards

It gives me that Link:

youtube-dl "https://dplaynordics-vod-59.akamaized.net/dplaydni/186694/0/hls/9966982004/playlist.m3u8?hdnts=st=1614517578~exp=1614603978~acl=/dplaydni/186694/0/hls/9966982004/*~hmac=904e05387a8ac2bda4677a459b9ccb57a2dc8e5718b53b145ba8b1132f25d3c2" -o "Yukon Men - S02E16 - Ein Dorf im Wandel.mp4"

but when i download it - it does not have audio:

frame=66650 fps= 68 q=-1.0 Lsize= 2068736kB time=00:44:25.88 bitrate=6357.0kbits/s
video:2067836kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.043528%
[ffmpeg] 2118385688 bytes
[download] 100% of 1.97GiB

how to fix this ?

With
python dmax.py 7163 -s 3

I get an error
Traceback (most recent call last):
File "dmax.py", line 232, in
chosen_episode=arguments.episode
File "dmax.py", line 170, in main
episode_name=episode.name
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 19: ordinal not in range(128)

I founded a list of show ids for dmax. I have added some entries. @Brawl345 Do you want the update ?

Hi there,

first of all, i'm not sure, if this some to fix on this side of the script.

The script sucessfully created an excel sheet with all download links, for example "Roter Stahl - Der Panzer Clan".
When i try to download via youtube-dl i get a http error 403: Forbidden.

I'm using the newest version of this script and of yt-download. The Github of YT-DL only says, maybe the Download is not wanted anymore.

Is there any solution, maybe by modif this script to get pass this?

Error Message:
youtube-dl.exe "https://dplaynordics-vod-06.akamaized.net/dplaydni/94222/0/hls/9064573004/playlist.m3u8?hdnts=st=1618338366~exp=1618424766~acl=/dplaydni/94222/0/hls/9064573004/*~hmac=08d1edfdfdf98c3d3266884a39cf8cc099e4aa98129ee33e8c229d270770e5ea" -o "Roter Stahl - Der Panzer-Clan - S01E06 - Abgetaucht!.mp4"
[generic] *~hmac=08d1edfdfdf98c3d3266884a39cf8cc099e4aa98129ee33e8c229d270770e5ea: Requesting header
WARNING: Could not send HEAD request to https://dplaynordics-vod-06.akamaized.net/dplaydni/94222/0/hls/9064573004/playlist.m3u8?hdnts=st=1618338366~exp=1618424766~acl=/dplaydni/94222/0/hls/9064573004/*~hmac=08d1edfdfdf98c3d3266884a39cf8cc099e4aa98129ee33e8c229d270770e5ea: HTTP Error 403: Forbidden
[generic] *~hmac=08d1edfdfdf98c3d3266884a39cf8cc099e4aa98129ee33e8c229d270770e5ea: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

No longer working for dmax.de

2020-10-19 21:47:13,055 - DMAX - INFO - Getting Authorization token for dmaxde...
2020-10-19 21:47:13,552 - DMAX - INFO - Getting show data
2020-10-19 21:47:13,849 - DMAX - INFO - => Steel Buddies
2020-10-19 21:47:13,851 - DMAX - INFO - Getting link 1 of 5
Traceback (most recent call last):
  File "dmax.py", line 263, in <module>
    realm=arguments.realm
  File "dmax.py", line 177, in main
    episode_name=episode.name
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 8: ordinal not in range(128)

Getting this issue since the last Steel Buddies episode with (updated): dmax.py 6023 -s 8 -e 5

Microsoft Defender blocked the exectuable today saying that it contains "Trojan:Win32/Trickbot!ml".