mosswg / dropout-dl Goto Github PK

After a fresh clone of this repo, the docker run command docker build -t dropout-dl:latest . resulted in an error:

To Reproduce
Steps to reproduce the behavior:

1. Clone the repo git clone https://github.com/mosswg/dropout-dl.git

C:\Code>git clone https://github.com/mosswg/dropout-dl.git
Cloning into 'dropout-dl'...
remote: Enumerating objects: 600, done.
remote: Counting objects: 100% (157/157), done.
remote: Compressing objects: 100% (110/110), done.
Receiving objects:  98% (588/600)sed 91 (delta 44), pack-reused 443 eceiving objects:  93% (558/600)
Receiving objects: 100% (600/600), 234.86 KiB | 2.33 MiB/s, done.
Resolving deltas: 100% (383/383), done.

2. cd into the new directory.
   C:\Code>cd dropout-dl
3. run docker build -t dropout-dl:latest .

C:\Code\dropout-dl>docker build -t dropout-dl:latest .
[+] Building 12.0s (10/13)                                                                                                                                                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                               0.1s
 => => transferring dockerfile: 521B                                                                                                                                                                                                                                                                               0.0s 
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                  0.1s 
 => => transferring context: 111B                                                                                                                                                                                                                                                                                  0.0s 
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                                                                                                                                                                                   1.0s
 => [1/9] FROM docker.io/library/alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48                                                                                                                                                                                                    0.8s
 => => resolve docker.io/library/alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48                                                                                                                                                                                                    0.1s 
 => => sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418 1.47kB / 1.47kB                                                                                                                                                                                                                     0.0s
 => => sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca 3.41MB / 3.41MB                                                                                                                                                                                                                     0.3s 
 => => sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48 1.64kB / 1.64kB                                                                                                                                                                                                                     0.0s
 => => sha256:13b7e62e8df80264dbb747995705a986aa530415763a6c58f84a3ca8af9a5bcd 528B / 528B                                                                                                                                                                                                                         0.0s 
 => => extracting sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca                                                                                                                                                                                                                          0.2s 
 => [internal] load build context                                                                                                                                                                                                                                                                                  0.2s 
 => => transferring context: 444.62kB                                                                                                                                                                                                                                                                              0.1s
 => [2/9] RUN echo "https://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories &&  echo "https://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories                                                                                                                          0.5s
 => [3/9] RUN apk update &&  apk add curl curl-dev alpine-sdk cmake make                                                                                                                                                                                                                                           8.4s
 => [4/9] WORKDIR /app                                                                                                                                                                                                                                                                                             0.1s
 => [5/9] COPY ./ /app                                                                                                                                                                                                                                                                                             0.1s
 => ERROR [6/9] RUN cmake -S /app -B build                                                                                                                                                                                                                                                                         0.9s
------
 > [6/9] RUN cmake -S /app -B build:
0.513 -- The C compiler identification is GNU 13.2.1
0.570 -- The CXX compiler identification is GNU 13.2.1
0.584 -- Detecting C compiler ABI info
0.673 -- Detecting C compiler ABI info - done
0.686 -- Check for working C compiler: /usr/bin/cc - skipped
0.687 -- Detecting C compile features
0.687 -- Detecting C compile features - done
0.694 -- Detecting CXX compiler ABI info
0.780 -- Detecting CXX compiler ABI info - done
0.799 -- Check for working CXX compiler: /usr/bin/c++ - skipped
0.799 -- Detecting CXX compile features
0.800 -- Detecting CXX compile features - done
0.802 CMake Error at CMakeLists.txt:61 (add_subdirectory):
0.802   The source directory
0.802
0.802     /app/external/json
0.802
0.802   does not contain a CMakeLists.txt file.
0.802
0.802
0.803 -- Configuring incomplete, errors occurred!
------
Dockerfile:13
--------------------
  11 |     COPY ./ /app
  12 |
  13 | >>> RUN cmake -S /app -B build
  14 |     WORKDIR /app/build
  15 |     RUN make && \
--------------------
ERROR: failed to solve: process "/bin/sh -c cmake -S /app -B build" did not complete successfully: exit code: 1

Expected behavior
The image should build successfully

Desktop (please complete the following information):

• OS: Windows 11
• Docker Desktop version 4.25.2 (129061)

Workaround
I was able to get around this by doing the following:

1. Update .gitmodules by changing the url parameter to https://github.com/nlohmann/json.git
2. Run git submodule sync
3. Run git submodule update --init --recursive

I hesitate to create a PR with this change because this feels like cumbersome extra steps, but @mosswg maybe you can find a more concise way to get this submodule pulled upon clone.

Describe the bug
When I was downloading the Breaking News series, I noticed that season 3 episode 26 got skipped. Looking at the page for that season, it appears that the season only shows the first 24 episodes and that the final episode is hidden until you click the Show More button at the bottom of the page. I assume that is why the scraper was unaware of it. I was able to download the episode on its own just fine as a temporary work around.

To Reproduce
Steps to reproduce the behavior:

1. run docker run --rm -it -v $PWD/login:/app/login -v $PWD/out:/Downloads dropout-dl:latest --output-directory /Downloads --captions -S https://www.dropout.tv/breaking-news-no-laugh-newsroom
2. check output/directory and see it skip from S03E25 to S04E01

Expected behavior
The Series finds and downloads all the episodes, even the hidden ones.

Screenshots

Describe the bug
The docker application asking me for login file even when login file exists.

To Reproduce
Steps to reproduce the behavior:
I've tried to follow the steps, compiled the docker, filled in the variables, filled in the login file.

Expected behavior
The docker application asking me for login file even when login file exists.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
Ubuntu 22.04.2 LTS"
Docker

Smartphone (please complete th

e following information):

Additional context
Thank you for doing this program/docker style. I would love for it to work.

Describe the bug
I think this is tied to the previous fix (#18), but it seems that newer episodes are now splitting audio into the separate m4a file (or, if it's an older file, just having a 0 size .m4a).

Anyway to have these get joined in the compile and spit out one combined file?

To Reproduce
Steps to reproduce the behavior:

1. Run
   docker run --rm -it -v $PWD/login:/app/login -v $PWD/out:/Downloads dropout-dl:latest --output-directory /Downloads --captions -e https://www.dropout.tv/dirty-laundry/season:3/videos/who-bugged-someone-s-car-to-catch-them-cheating
2. Observe the output files, and notice the .m4a has no audio track.
3. Older videos don't seem to have this issue.

Expected behavior
Combined a/v files as final output, and no .m4a needed.

Desktop:

• OS: Mac OSX 14.0
• Browser Chrome, but using login file

Describe the bug
When downloading a whole series, such as Make Some Noise, if the download runs a few times, it eventually downloads season 1 again, as season 10. It places the files in the season 1 folder but names them s10Exx, and season 2 is downloaded as 22,

To Reproduce
Steps to reproduce the behavior:

1. Download a series via cron hourly using docker run --name dropout --rm -it -v $PWD/login:/app/login -v /mnt/nas/video/dropout:/Downloads dropout-dl:latest --output-directory /Downloads --captions -q 1080p https://www.dropout.tv/$series
2. Leave it running for a day or so, and it eventually downloads duplicates.

System

• OS: Ubuntu, but running the app inside Docker
• built from 6014531

Some videos use the language "en-US" instead of "en" and sometimes the type is "subtitles" instead of "captions". This causes DDL to miss some captions. A way to check for all of these possibilities would solve this.

Describe the bug
When I attempt to download a single-episode Dropout special, I get an Assertion error

To Reproduce

Input:

docker run --rm -it -v $PWD/login:/app/login -v $PWD/out:$HOME/Downloads dropout-dl:latest -v --output-directory $HOME/Downloads -e https://
www.dropout.tv/new-releases/videos/the-legendary-rick-perry-and-the-art-of-dimension-20

Output:

quality: highest
verbose: 1
url: "https://www.dropout.tv/new-releases/videos/the-legendary-rick-perry-and-the-art-of-dimension-20"
Got episode url: https://www.dropout.tv/new-releases/videos/the-legendary-rick-perry-and-the-art-of-dimension-20 from program arguments
Logging in...
Successfully logged in.
Cached tokens to /root/.cache/dropout-dl/token-cache
Getting episode
Got page data
Got episode metadata: {"PROPERTIES":{"CANONICAL_COLLECTION":{"href":"https://www.dropout.tv/the-legendary-rick-perry-and-the-art-of-dimension-20","id":955952,"name":"The Legendary Rick Perry and the Art of Dimension 20"},"COLLECTION_ID":129054,"COLLECTION_TITLE":"New Releases","PRODUCT_ID":28599,"VIDEO_ID":3012752,"VIDEO_TITLE":"The Legendary Rick Perry and the Art of Dimension 20","VIEW_TYPE":"video"}}
Got name: The Legendary Rick Perry and the Art of Dimension 20
Assertion failed: it != m_data.m_value.object->end() (/app/external/json/include/nlohmann/json.hpp: operator[]: 2142)

Desktop (please complete the following information):
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

Linux Cyclone 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Starting last week, new episodes of Dimension 20 fail to download. Likely a format change on Dropout's end for new videos?

When trying last week or this week's episode:
quality: 360p
verbose: 1
url: "https://www.dropout.tv/dimension-20-burrow-s-end/season:1/videos/protect-the-light"
Got episode url: https://www.dropout.tv/dimension-20-burrow-s-end/season:1/videos/protect-the-light from program arguments
Using cached tokens from REDACTED
Getting episode
Got page data
Got episode metadata: "PROPERTIES":{"VIEW_TYPE":"video","VIDEO_ID":2947656,"COLLECTION_ID":934445,"COLLECTION_TITLE":"Season 1","PRODUCT_ID":28599,"VIDEO_TITLE":"Protect the Light","CANONICAL_COLLECTION":{"id":934445,"name":"Season 1","href":"https://www.dropout.tv/season-1-109","parent":{"id":934442,"name":"Dimension 20: Burrow's End","type":"series"}}}
Got name: Protect the Light
Got series: Dimension 20: Burrow's End
Got season: Season 1
Got series directory: Dimension 20 Burrow's End
Got embedded url: https://embed.vhx.tv/videos/2947656?api=1&auth-user-token= REDACTED&autoplay=1&back=Dimension%2020%3A%20Burrow%27s%20End%20-%20Season%201&collection_id=934445&color=feea3b&context=https%3A%2F%2Fwww.dropout.tv%2Fdimension-20-burrow-s-end%2Fseason%3A1&is_trailer=false&live=0&locale=en&playsinline=1&product_id=28599&referrer=&sharing=1&title=0&vimeo=1
Got config url: https://player.vimeo.com/video/878429186/config?autoplay=1&color=feea3b&speed=1&token= REDACTED&trick_play=1
filename:
ERROR: quality of 360p not found

Today, trying an episode uploaded in previous weeks:
quality: 360p
verbose: 1
url: "https://www.dropout.tv/dimension-20-burrow-s-end/season:1/videos/a-second-sun"
Got episode url: https://www.dropout.tv/dimension-20-burrow-s-end/season:1/videos/a-second-sun from program arguments
Using cached tokens from REDACTED
Getting episode
Got page data
Got episode metadata: "PROPERTIES":{"VIEW_TYPE":"video","VIDEO_ID":2930807,"COLLECTION_ID":934445,"COLLECTION_TITLE":"Season 1","PRODUCT_ID":28599,"VIDEO_TITLE":"A Second Sun","CANONICAL_COLLECTION":{"id":934445,"name":"Season 1","href":"https://www.dropout.tv/season-1-109","parent":{"id":934442,"name":"Dimension 20: Burrow's End","type":"series"}}}
Got name: A Second Sun
Got series: Dimension 20: Burrow's End
Got season: Season 1
Got series directory: Dimension 20 Burrow's End
Got embedded url: https://embed.vhx.tv/videos/2930807?api=1&auth-user-token= REDACTED&autoplay=1&back=Dimension%2020%3A%20Burrow%27s%20End%20-%20Season%201&collection_id=934445&color=feea3b&context=https%3A%2F%2Fwww.dropout.tv%2Fdimension-20-burrow-s-end%2Fseason%3A1&is_trailer=false&live=0&locale=en&playsinline=1&product_id=28599&referrer=&sharing=1&title=0&vimeo=1
Got config url: https://player.vimeo.com/video/875673565/config?autoplay=1&color=feea3b&speed=1&token= REDACTED&trick_play=1
Found quality (7620 + 4): 720p
Found url (7318 + 261): https://vod-progressive.akamaized.net/exp=1698928078~acl=%2Fvimeo-prod-skyfire-std-us%2F01%2F134%2F35%2F875673565%2F3669808373.mp4~hmac=024072c0598f880b2f78fe3370f084f87b561497dccd4133ff005bb7d2dd7107/vimeo-prod-skyfire-std-us/01/134/35/875673565/3669808373.mp4
Found quality (8068 + 4): 240p
Found url (7766 + 261): https://vod-progressive.akamaized.net/exp=1698928078~acl=%2Fvimeo-prod-skyfire-std-us%2F01%2F134%2F35%2F875673565%2F3669808372.mp4~hmac=81aecf0d27ef4c2fe4c9876486ebef84602d7af675df682332c910122e5545c7/vimeo-prod-skyfire-std-us/01/134/35/875673565/3669808372.mp4
Found quality (8516 + 4): 360p
Found url (8214 + 261): https://vod-progressive.akamaized.net/exp=1698928078~acl=%2Fvimeo-prod-skyfire-std-us%2F01%2F134%2F35%2F875673565%2F3669814631.mp4~hmac=4df4536d736d6014d8c8ecc5b528ad0dabf1ee6d2515a0e4bfed52d571cf970c/vimeo-prod-skyfire-std-us/01/134/35/875673565/3669814631.mp4
Found quality (8964 + 4): 540p
Found url (8662 + 261): https://vod-progressive.akamaized.net/exp=1698928078~acl=%2Fvimeo-prod-skyfire-std-us%2F01%2F134%2F35%2F875673565%2F3669808375.mp4~hmac=489b550e13210f0c397c3d8b230ad9a7056c3b35c18eebc4c4823817406f9143/vimeo-prod-skyfire-std-us/01/134/35/875673565/3669808375.mp4
Found quality (9414 + 5): 1080p
Found url (9112 + 261): https://vod-progressive.akamaized.net/exp=1698928078~acl=%2Fvimeo-prod-skyfire-std-us%2F01%2F134%2F35%2F875673565%2F3669808374.mp4~hmac=03cfc02116d55e65ad4b543b53a2c2ad9e3787b85f0f16912f8a059244c3589b/vimeo-prod-skyfire-std-us/01/134/35/875673565/3669808374.mp4
filename:
Dimension 20 Burrow's End - A Second Sun [▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 51MiB / 627MiB

Describe the bug
When running dropout-dl, the application is able to take the URL and determine the episode listings, but is unable to actually download. However, when running the application with a single episode link it has no trouble.

To Reproduce
Run command: ./dropout-dl --output-directory "/mnt/plexcontent/TV Shows/" --series https://www.dropout.tv/game-changer

Expected behavior
Media downloaded

Screenshots

Episode Example:

Desktop (please complete the following information):
Distributor ID: Ubuntu
Description: Ubuntu 22.10
Release: 22.10
Codename: kinetic

I'm very new to Docker, so sorry in advance if this is a basic issue.

I've been trying to run the example command from the description, with adjusted outputs, but the only response I get is
PS C:\Users\USER> docker run --rm -it -v /login:/app/login -v /downloads dropout-dl:latest --output-directory /downloads --captions -e https://www.dropout.tv/dimension-20-mentopolis Logging in... ERROR: Invalid login format in file. File must contain just your email then password on seperate lines. Example: [email protected] password123

I tried just running the Docker in Docker Desktop, but I haven't figured out how to add options and variables when running it so it just exits immediately with this in the log
2023-09-18 22:19:12 Enter episode url: Logging in... 2023-09-18 22:19:12 Successfully logged in. Cached tokens to /root/.cache/dropout-dl/token-cache 2023-09-18 22:19:12 SERIES PARSE ERROR: Could not parse series name 2023-09-18T20:19:12.746979236Z

Is your feature request related to a problem? Please describe.
When no video quality is specified the code assumes 1080p.

For some older series (for example: Badman) there is no 1080p quality available. The container will exit after discovering the episodes but without downloading.

Describe the solution you'd like
When no quality is specified by the user, check at least the first episode (ideally for each episode) and select the highest available quality for download.

Describe alternatives you've considered
You can inspect the container log and then manually adjust your runtime command to use the highest quality but this isn't very automation friendly.

Is your feature request related to a problem? Please describe.
I downloaded some videos before realizing that it was possible to download captions with them, so I don't want to have to download the videos all over again with the captions.

Describe the solution you'd like
A simple flag to only download captions and skip downloading the video.

It says cookies cannot be used because SQLite is not installed, but it is. I installed the sqlite3 package using apt prior to building, but it still fails to retrieve cookies. Did I install the wrong package or is something else going wrong here?

Describe the bug
When downloading a season (or its individual episodes), sometimes a wildly incorrect season number is used in the filename.

To Reproduce
Run command docker run --rm -it -v $PWD/login:/app/login -v $PWD/out:/Downloads dropout-dl:latest --output-directory /Downloads --captions --verbose -q 1080p -s https://www.dropout.tv/dimension-20/season:5
This is intermittent does not happen with all seasons of all shows. Anecdotally, it appears that for every season that this happens, logs will show many instances of the message "ERROR: Could not find start of substring" when parsing for episodes.

Expected behavior
File names should have S05EXX. In my case, they had S52EXX

Desktop (please complete the following information):
Windows 11

Possibly related to #5