kamui-fin / yt-to-anki Goto Github PK

View Code? Open in Web Editor NEW

20.0 2.0 3.0 243 KB

Simple anki addon to generate decks from youtube videos

Home Page: https://ankiweb.net/shared/info/964531817

License: GNU General Public License v3.0

Python 100.00%

anki subs2srs ytdl subtitles language-learning

yt-to-anki's Introduction

YouTube to Anki

An Anki addon that generates a full-fledged deck with audio and pictures from a YouTube link using subtitles with the click of a button. This eliminates the need for having to download a video, or the accompanying subtitles, re-timing them, and dealing with TSV files to import. For those of you familiar with Subs2SRS, consider this to be the YouTube version.

This addon offers several features:

Supports all languages
Fallback to automatically generated subs if man-made captions could not be found
Set a limit to how many cards are generated
Choose the dimensions of the pictures
Fast card generation

Installation

In Tools > Addons, click Get addon and use the code 964531817
Restart Anki for changes to take place

If you are a Linux or Mac user, make sure to install ffmpeg using your package manager.

Usage

Enter the YouTube link for the video
Set an output folder for the audio and screenshots to go (can also be used for condensed audio)
Choose the appropriate note type and fields for the card data
Specify the subtitle language (default: English)
Hit generate. After a bit, refresh your decks, and you should see a deck named after the title of the video

Quality of the subtitles

TL;DR: In order to get the best learning experience, work with the YouTube's videos that have high-quality manually generated subtitles (e.g., TED talks). Enable option "Optimize subtitles" to get sentence-based Anki cards: one card per sentence.

The YouTube to Anki (Y2A) extension uses subtitle ranges to cut the corresponding audio fragments out of the downloaded video. If the subtitles are well-crafted by the creators of a YouTube video, the resulting Anki cards will 90%+ perfectly match the audio from the MP3 files generated by the extension.

YouTube's videos can have automatically and manually generated subtitles.

When a video has manually generated subtitles, it is likely to have:

Capitalized sentences and correct punctuation.
Subtitle ranges matching the actual sentences as they are spoken on a video.

For the manually generated subtitles, the Y2A extension provides an optimization which merges the subtitle texts so that the full sentences are formed, one sentence per one Anki card. The optimization relies on the punctuation found in the subtitles, e.g. ".", ",", "?" in order to decide when a sentence should end. The option is enabled with the "Optimize subtitles" flag.

Examples of good YouTube videos that can be used for testing this extension:

When a video has automatically generated subtitles, it is most often that the extension cannot produce good Anki cards: the subtitle text will have neither capitalized letters nor punctuation, and therefore the Anki cards will not be sentence-based.

Development

Before moving on, be sure you have python and poetry setup.

Clone the repository

git clone https://github.com/kamui-fin/yt-to-anki.git
cd yt-to-anki

Install dependencies

poetry install

Install pre-commit hooks

poetry run pre-commit install

If you are on windows, you need to create a top-level directory called ffmpeg with ffmpeg.exe inside
Bundle everything (or update the bundle)

poetry run invoke package-dev

Create a symlink of dist/ inside of your Anki data's addons21/ folder

ln -s ./dist ~/.local/share/Anki2/addons21/yt-to-anki

Now you can use the dev command for simultaneously running anki with the updated bundle

poetry run invoke dev

Contributing

All contributions are gladly welcomed! Feel free to open an issue or create a pull request if you have any new changes/ideas in mind.

yt-to-anki's People

Contributors

Stargazers

Watchers

Forkers

haohao11 shoudairika delozi

yt-to-anki's Issues

Maintain a pyproject.toml or a requirements.txt file to track development dependencies

The development dependencies are at least:

invoke (developer tasks)
black (formatting)
pytest (unit testing)
ruff (Python linter)

It would be great to be able to install all them with one command-line command.

The extension crashes when reopening the app and while trying to read the stored settings

Steps to reproduce:

Create a video with somewhat normal settings.
Close the app.
Open the app.
Click to open the extension in the Extras.

Fehler
Es ist ein Fehler aufgetreten. Bitte starten Sie Anki neu und halten Sie während des Startvorgangs die Umschalttaste gedrückt, damit Anki die installierten Erweiterungen temporär deaktiviert.
Wenn das Problem nur auftritt, wenn die Erweiterungen aktiviert sind, dann benutzen Sie bitte den Menueintrag Extras > Erweiterungen um die problembehaftete Erweiterung durch selektives Deaktivieren herauszufinden.
Wenn Sie die entsprechende Erweiterung herausgefunden haben, dann bitten wir, dass Sie sich mit einem entsprechenden Problembericht an die [Support-Seite für Erweiterungen](https://forums.ankiweb.net/c/add-ons/11) wenden, damit der Fehler behoben werden kann.
Debuginformationen:
Anki 2.1.57 (6b7d372c) Python 3.11.1 Qt 5.15.2 PyQt 5.15.9
Platform: macOS-12.6.2-x86_64-i386-64bit
Flags: frz=False ao=True sv=2
Add-ons, last update check: 2023-02-14 22:05:51
Möglicherweise beteiligte Erweiterungen: ⁨964531817⁩

Caught exception:
Traceback (most recent call last):
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/main.py", line 164, in launch
    screen.setup_ui()
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/main.py", line 74, in setup_ui
    self.read_settings()
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/main.py", line 37, in read_settings
    self.width_box.setValue(self.settings.value("width", 240))
TypeError: setValue(self, val: int): argument 1 has unexpected type 'str'

Auto-fill or memorize as many UI fields as possible

It would be great if I had less fields to type in before I press generate.

For example, the output folder for me is always: /tmp. And the language is for me most of the times German.

How about preserving the last input settings in a file somewhere? Does Anki have a native API for doing this kind of thing maybe?

Cannot import name 'launch' from 'main'

When loading '⁨yt2srs⁩':
⁨Traceback (most recent call last):
File "/home/yaoberh/pyenv/lib/python3.9/site-packages/aqt/addons.py", line 239, in loadAddons
import(addon.dir_name)
File "/home/yaoberh/.local/share/Anki2/addons21/964531817/init.py", line 8, in
from main import launch
ImportError: cannot import name 'launch' from 'main' (/home/yaoberh/.local/share/Anki2/addons21/1186808928/main.py)

Why do I have ffmpeg installed on my mac and it still doesn't work

mp3 file not found when generating deck

After having some initial problems with ffmpeg not being found on Mac (the addon doesn't check /opt/homebrew/bin/ffmpeg where ffmpeg will be installed using Brew), I was unable to run the addon using the following video: https://www.youtube.com/watch?v=mapbKTJ9aBs

Error log below:

Error 
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed. 
If the issue only occurs when add-ons are enabled, please use the Tools>Add-ons menu item to disable some add-ons and restart Anki, repeat until you discover the add-on that is causing the problem. 
When you've discovered the add-on that is causing the problem, please report the issue on the add-ons section of our support site. 
Debug info:
Anki 2.1.65 (aa9a734f) Python 3.9.15 Qt 6.5.0 PyQt 6.5.0
Platform: macOS-14.0-arm64-arm-64bit
Flags: frz=True ao=True sv=3
Add-ons, last update check: 2023-12-28 12:00:30
Add-ons possibly involved: ⁨Youtube to Anki⁩

Caught exception:
Traceback (most recent call last):
  File "/Users/anawfulfeeling/Library/Application Support/Anki2/addons21/964531817/worker.py", line 166, in add_card
    audiofname = mw.col.media.addFile(subtitle_range.audio_path)
  File "anki.media", line 94, in add_file
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/5z/kwxwbktd1w9_jn75jrs99fvm0000gp/T/How to Use じゃない - ja-orig_                  1900-01-01 00_00_00_240000_                  3_429.mp3'

Caught exception:
Traceback (most recent call last):
  File "/Users/anawfulfeeling/Library/Application Support/Anki2/addons21/964531817/worker.py", line 166, in add_card
    audiofname = mw.col.media.addFile(subtitle_range.audio_path)
  File "anki.media", line 94, in add_file
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/5z/kwxwbktd1w9_jn75jrs99fvm0000gp/T/How to Use じゃない - ja-orig_                  1900-01-01 00_00_03_679000_                  3_311.mp3'

Caught exception:
Traceback (most recent call last):
  File "/Users/anawfulfeeling/Library/Application Support/Anki2/addons21/964531817/worker.py", line 166, in add_card
    audiofname = mw.col.media.addFile(subtitle_range.audio_path)
  File "anki.media", line 94, in add_file
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/5z/kwxwbktd1w9_jn75jrs99fvm0000gp/T/How to Use じゃない - ja-orig_                  1900-01-01 00_00_06_990000_                  0_01.mp3'

Feature request: Progress indication when downloading YouTube video

The youtube-dl client supports hooks. With a hook, it should be possible to get the info about the progress and report it to the user interface.

See https://stackoverflow.com/a/58667850/598057.

Add-on failed to load

I just upgraded to the newest anki Version ⁨23.12.1 (1a1d4d54)⁩ Python 3.9.15 Qt 6.5.3 PyQt 6.5.3 and installed the addon, but I get add-on failed to load when I open anki. Debug info below.

Anki 23.12.1 (1a1d4d54) (ao)
Python 3.9.15 Qt 6.5.3 PyQt 6.5.3
Platform: macOS-14.0-x86_64-i386-64bit

When loading Youtube to Anki:
Traceback (most recent call last):
File "aqt.addons", line 245, in loadAddons
File "/Users/matt/Library/Application Support/Anki2/addons21/964531817/init.py", line 9, in
from .main import launch
File "/Users/matt/Library/Application Support/Anki2/addons21/964531817/main.py", line 3, in
from PyQt5 import QtCore, QtWidgets
ModuleNotFoundError: No module named 'PyQt5'

Migrate to QT6

Anki is being ditributed primarly with Qt6 now :
https://docs.ankiweb.net/platform/linux/installing.html#qt5-vs-qt6

yt-to-anki doesn't work with the Qt6 version.

Maybe start with a warning in the README

DeprecationWarning: invalid escape sequence '\d'

  /Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/client_youtube.py:160: DeprecationWarning: invalid escape sequence '\d'
    "(\d+:\d+:\d+\.\d+) --> (\d+:\d+:\d+\.\d+).*\n", chunk

Error in the production version of the extension: No module named "fileinput"

Error when trying to generate deck

After fulfilling all the necessary fields, I keep getting an error that pops up when I hit the generate button. I tried generating with/without the "fallback to auto gen subs" on, but it still doesn't work. Once the Anki Error dialog box pops up, I am left with an infinite loading bar.

Debug Information

Error An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed. If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem. When you've discovered the add-on that is causing the problem, please report the issue on the add-on support site. Debug info: Anki 2.1.23 (de9543ff) Python 3.8.0 Qt 5.13.1 PyQt 5.14.1 Platform: Mac 10.15.5 Flags: frz=True ao=True sv=2 Add-ons, last update check: 2020-06-23 06:59:00 Add-ons possibly involved: ⁨YT Subs2SRS⁩

Caught exception:
Traceback (most recent call last):
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/YoutubeDL.py", line 797, in extract_info
ie_result = ie.extract(url)
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/extractor/youtube.py", line 2762, in _real_extract
has_videos, playlist = self._extract_playlist(playlist_id)
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/extractor/youtube.py", line 2685, in _extract_playlist
raise ExtractorError(message, expected=True)
youtube_dl.utils.ExtractorError: This playlist is private, use --username or --netrc to access it.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/yt_parse.py", line 91, in run
ydl.download([self.link])
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/YoutubeDL.py", line 2018, in download
res = self.extract_info(
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/YoutubeDL.py", line 820, in extract_info
self.report_error(compat_str(e), e.format_traceback())
File "/Users/chris/Library/Application Support/Anki2/addons21/1633834851/youtube_dl/YoutubeDL.py", line 620, in report_error
if not self.params.get('no_color') and self._err_file.isatty() and compat_os_name != 'nt':
AttributeError: 'ErrorHandler' object has no attribute 'isatty'

Error handling when downloading YouTube videos and subtitles: Detect if a video / subtitles have been downloaded or failed

Currently, if a video only has auto-generated subtitles and the fallback option is NOT enabled, the extension crashes with the following trace:

It should be possible to do full error handling when youtube-dl fails to download content. The Hooks are likely a way to go.

The issue #5 seems to be directly related.

Deprecation warnings: please use 'set_current', 'by_name', 'add_file'

addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:124:byName is deprecated: please use 'by_name'
addons21/964531817/worker.py:127:setCurrent is deprecated: please use 'set_current'
addons21/964531817/worker.py:132:addFile is deprecated: please use 'add_file'
addons21/964531817/worker.py:137:addFile is deprecated: please use 'add_file'

Feature: System readiness check screen

From #7:

A separate screen could show if a user has all necessary dependencies installed. At least, I have the ffmpeg and the mpv player in mind. This way the UI could present nicely if everything should work or user should still take some action.

Under the hood, the extension could run ffmpeg -v or mpv -v and ensure that some expected output is printed. Otherwise, the screen could suggest an action to a user.

IndexError: list index out of range

This error happens immediately after the video has downloaded:

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue on the add-on support site.
Debug info:
Anki 2.1.48 (fb07bad3) Python 3.9.7 Qt 5.15.2 PyQt 5.15.4
Platform: Linux
Flags: frz=False ao=True sv=3
Add-ons, last update check: 2021-10-22 21:14:42
Add-ons possibly involved: ⁨yt2srs⁩

Caught exception:
Traceback (most recent call last):
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/worker.py", line 89, in run
    dL_sources = (glob(sub_dir + "/*")[0], glob(vid_dir  + "/*")[0], title)
IndexError: list index out of range

A word of thanks and a check-in on the maintenance status

Hi @kamui-fin,

Thanks for creating this extension. It is easy to use and should very well serve the needs of many language learners.

My main question: are you planning to maintain it in the future?

My story goes back to my lessons with my German language teacher, who actively uses Anki in all of his lessons. He has found your extension and wanted to try it. He could not create a deck from the YouTube videos that he tried, and he asked me if I could take a look into what was happening. I tried it as well and could not generate any deck either (my machine is macOS, my teacher has Windows).

I have cloned Anki and forked your repository, integrated your extension into Anki and started debugging it in the IDE. I have found a few issues and some of them I could already fix, but some of them require further work.

Known issues

You bundle the extension with FFmpeg which is a great idea, but it creates a problem of having to maintain the ffmpeg executable binaries for at least Windows, Linux, and macOS. The bundled non-Windows binary didn't work in macOS terminal, so I had to install FFmpeg manually with brew install ffmpeg and remove the binaries from the extension folder. I would assume that a user could be asked to install FFmpeg/mpv and have these tools in their PATH variable.
I several times encountered the issue reported here: #1 and then I figured out that you can silence but not fix this issue by setting "no_warnings": True, in the YouTube options. That issue seems to be related to the fact that Anki is patching sys.stderr with a custom error handler which does not support istty() and flush() methods. This issue needs further investigation and improved error handling because otherwise Anki UI gets stuck on the progress screen with no feedback reported to a user.
If a video conversion fails and for some reason the previous video is not removed, the ffmpeg fails with the file already exists. do you want to overwrite it? (y/n) leaving the Anki app hanging with no further progress. I have fixed this for now by adding -y to the ffmpeg commands. Also, the algorithm of making glob[0]... to find a downloaded video/subtitles seems to be too broad and could be reduced to avoid any collisions between the files that can also be in the folder.
The subtitles are parsed correctly when they are not auto-generated. When they are, the subtitles files contain more structure and lines with <c>...</c> tags that have to be recognized by your parser. This is something that I could consider fixing later on.
The code was missing formatting, linting, unit tests, and I felt like I could take it to the next level and added a few basic quality artifacts.

Future work

Some YouTube videos take considerable time to be downloaded. The youtube-dl supports a progress hook which could be used for presenting the progress indication on the User Interface.
A separate screen could show if a user has all necessary dependencies installed. At least, I have the ffmpeg and the mpv player in mind. This way the UI could present nicely if everything should work or user should still take some action.
When the subtitles are manually created and well-written, they can be merged together to form full sentences so that the audio files are cut sentence-by-sentence not part-by-part. This can result in a very precise cutting and therefore in a much better user experience out of the box. For automatically generated subtitles, at least a distance check can be made: If the subtitles are very close in time to each other, they can be also merged because it's likely that they are part of the same sentence spoken in the video. Both algorithms could be enabled with dedicated option checkboxes in the UI.

You can see my changes in the forked repo: https://github.com/stanislaw/yt2srs

My dilemma is as follows: I would like to fix the most pressing issues and release an updated extension for my teacher. Are you willing to accept all my changes via Pull Requests and update the extension, or should I go completely independently with my fork and create a new extension which I and my teacher could maintain ourselves?

In case you would like to participate in this, could I count on your assistance and readiness to support us with timely releases of the package? I am rather inclined to continue with a fork, but I would be happy to contribute all changes back to your main repository if you have interest / resources to review all the changes and especially release the updated extension.

Please let me know what you think.

Smarter subtitles cutting optimization for automatically and not automatically generated subtitles

This issue is a follow-up to the other subtitles-related issue: #22.

I have found that with both auto- and non-auto subtitles it is very often the case that the subtitles are cut in the middle of a sentence and the resulting mp3 sounds very incomplete. It looks like two possible optimizations can be applied in order to cut the cards sentence-based. This can be ensured by merging the subtitles that follow each other closely.

The original description can be found here: #7.

When the subtitles are manually created and well-written, they can be merged together to form full sentences so that the audio files are cut sentence-by-sentence not part-by-part. This can result in a very precise cutting and therefore in a much better user experience out of the box. For automatically generated subtitles, at least a distance check can be made: If the subtitles are very close in time to each other, they can be also merged because it's likely that they are part of the same sentence spoken in the video. Both algorithms could be enabled with dedicated option checkboxes in the UI.

Not automatically generated subtitles

This is a very good example of very well crafted subtitles: https://www.youtube.com/watch?v=GfF2e0vyGM4. Each of the subtitle forms a part of a sentence and at some point the sentence finishes with a good punctuation sign like . or ?. In this case, it can be optimal to concatenate the subtitles until one sentence is full finished. This should make perfect Anki cards, one card per sentence.

Automatically generated subtitles

In this case, it is not possible to rely on punctuation. Instead, one subtitle range can be checked with a previous one:

If the distance between subtitle ranges is less than 0.1s, they can be concatenated as forming the same sentence. Otherwise, they are two different cards and should be kept separate.

Using this technique, I could achieve a decent cutting of the sentences as spoken in this video: https://www.youtube.com/watch?v=7I4J4vy2Deo&t=12s.

Both techniques could be activated with a corresponding option in the UI.

Randomly timed Isatty error

This error happens sometimes just after clicking start, sometimes 5 minutes into downloading the video:

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue on the add-on support site.
Debug info:
Anki 2.1.48 (fb07bad3) Python 3.9.7 Qt 5.15.2 PyQt 5.15.4
Platform: Linux
Flags: frz=False ao=True sv=3
Add-ons, last update check: 2021-10-22 21:14:42
Add-ons possibly involved: ⁨yt2srs⁩

Caught exception:
Traceback (most recent call last):
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 1970, in process_info
    partial_success = dl(fname, new_info)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 1915, in dl
    return fd.download(name, info)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/downloader/http.py", line 351, in real_download
    establish_connection()
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/downloader/http.py", line 116, in establish_connection
    raise err
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/downloader/http.py", line 110, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 2288, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/worker.py", line 87, in run
    ydl.download([self.link])
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 2068, in download
    res = self.extract_info(
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 808, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 847, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 881, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 1692, in process_video_result
    self.process_info(new_info)
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 1978, in process_info
    self.report_error('unable to download video data: %s' % error_to_compat_str(err))
  File "/home/yannicko/.local/share/Anki2/addons21/964531817/youtube_dl/YoutubeDL.py", line 623, in report_error
    if not self.params.get('no_color') and self._err_file.isatty() and compat_os_name != 'nt':
AttributeError: 'ErrorHandler' object has no attribute 'isatty'

Error Creating Deck

Thank you for creating this addon, i really appreciate it! However, no matter what i do, an error comes up when trying to create a deck. I've read your responses to the other 2 versions of this issue, and i am using the updated version of the addon found at https://ankiweb.net/shared/info/964531817.

video link: https://www.youtube.com/watch?v=M5X_Ijxm2bw

Debug info:
Anki 2.1.35 (84dcaa86) Python 3.8.0 Qt 5.14.2 PyQt 5.14.2
Platform: Windows 10
Flags: frz=True ao=True sv=1
Add-ons, last update check: 2021-08-19 06:33:55

Caught exception:
Traceback (most recent call last):
File "C:\Users<redacted>\AppData\Roaming\Anki2\addons21\964531817\main.py", line 28, in
self.generate_button.clicked.connect(lambda : self.generate())
File "C:\Users<redacted>\AppData\Roaming\Anki2\addons21\964531817\main.py", line 77, in generate
parser.create_subs2srs_deck(**self.config)
AttributeError: module 'parser' has no attribute 'create_subs2srs_deck'

Still not working in Anki 2.1.60

It doesn't generate the deck we want to. Moreover, it doesn't support Japanese, after I have tried English (somehow the cards end up in other fixed deck. Note: it is not default.)

Anki does not show the plugin anymore after the "Refactoring of the root module" commit

This commit e2264d2, refactor: root ytsrs module seems to have broken some Anki convention: I no longer see the yt2srs in the Extras menu.

Does this have to do with a root __init__.py that is no longer there? How did you test this to work?

Man-made subtitles not being recognized as such, fallback to auto-generated subs throws error

The man-made Simplified Chinese subs for this video are not being recognized as such, and when the "fallback to auto-generated subs" option is turned on, the error copied from Anki and listed below is thrown:

https://www.youtube.com/watch?v=1kKJNLS4l1Q

( The same error occurs for this video in Traditional Chinese
https://www.youtube.com/watch?v=eoFLFsklB88 )

ERROR

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue to the add-on author.
Debug info:
Anki 2.1.60 (76d88073) Python 3.9.15 Qt 6.4.2 PyQt 6.4.0
Platform: Windows-10-10.0.19045
Flags: frz=True ao=True sv=2
Add-ons, last update check: 2023-03-11 14:00:00

Caught exception:
Traceback (most recent call last):
File "C:\Users\sg\AppData\Roaming\Anki2\addons21\964531817\worker.py", line 81, in run
result: YouTubeDownloadResult = YouTubeClient.download_video_files(
File "C:\Users\sg\AppData\Roaming\Anki2\addons21\964531817\client_youtube.py", line 40, in download_video_files
path_to_subtitles_file = glob(video_task.subtitle_path + "/*")[0]
IndexError: list index out of range

Extend subtitles parser to recognize the format of automatically generated subtitles

The subtitles are parsed correctly when they are not auto-generated (and we already have a working unit test for this 🥳 ).

When they are auto-generated, the subtitles files contain more structure and lines with funny <c>...</c> tags that have to be recognized by the parser.

Here is an example:

WEBVTT
Kind: captions
Language: de

00:00:00.000 --> 00:00:02.869 align:start position:0%
 
hallo <00:00:00.430><c>ich </c><00:00:00.860><c>bin </c><00:00:01.290><c>david </c><00:00:01.720><c>gründer </c><00:00:02.150><c>der </c><00:00:02.580><c>lingus</c>

00:00:02.869 --> 00:00:02.879 align:start position:0%
hallo ich bin david gründer der lingus
 

00:00:02.879 --> 00:00:05.390 align:start position:0%
hallo ich bin david gründer der lingus
organic <00:00:03.336><c>und </c><00:00:03.793><c>erfinder </c><00:00:04.250><c>der </c><00:00:04.707><c>bilingue</c>

Seems directly relevant:

https://gist.github.com/davidcortesortuno/64723e4262889f592def55c1927db651

YouTube downloader bug: Unable to extract uploader id

It looks like yt-dlp has an issue that was already reported: yt-dlp/yt-dlp#6253.

For any video that I am trying to download, I am getting the Unable to extract uploader id error (see below). This comment suggests that the fix is on the way.

One thought about this: how about we pin all the production Python package versions for now?

Fehler
Es ist ein Fehler aufgetreten. Bitte starten Sie Anki neu und halten Sie während des Startvorgangs die Umschalttaste gedrückt, damit Anki die installierten Erweiterungen temporär deaktiviert.
Wenn das Problem nur auftritt, wenn die Erweiterungen aktiviert sind, dann benutzen Sie bitte den Menueintrag Extras > Erweiterungen um die problembehaftete Erweiterung durch selektives Deaktivieren herauszufinden.
Wenn Sie die entsprechende Erweiterung herausgefunden haben, dann bitten wir, dass Sie sich mit einem entsprechenden Problembericht an die [Support-Seite für Erweiterungen](https://forums.ankiweb.net/c/add-ons/11) wenden, damit der Fehler behoben werden kann.
Debuginformationen:
Anki 2.1.57 (6b7d372c) Python 3.11.1 Qt 5.15.2 PyQt 5.15.9
Platform: macOS-12.6.2-x86_64-i386-64bit
Flags: frz=False ao=True sv=2
Add-ons, last update check: 2023-02-16 20:02:40
Möglicherweise beteiligte Erweiterungen: ⁨964531817⁩

ERROR: [youtube] FVGS1pgaZQ4: Unable to extract uploader id; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
Caught exception:
Traceback (most recent call last):
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1502, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1578, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 680, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/extractor/youtube.py", line 4049, in _real_extract
    'uploader_id': self._search_regex(r'/(?:channel|user)/([^/?&#]+)', owner_profile_url, 'uploader id') if owner_profile_url else None,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1228, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
yt_dlp.utils.RegexNotFoundError: [youtube] FVGS1pgaZQ4: Unable to extract uploader id; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/worker.py", line 75, in run
    result: YouTubeDownloadResult = YouTubeClient.download_video_files(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/client_youtube.py", line 47, in download_video_files
    YouTubeClient._download_subtitles(
  File "/Users/stanislaw/Library/Application Support/Anki2/addons21/964531817/client_youtube.py", line 97, in _download_subtitles
    ydl.download([video_task.youtube_video_url])
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3358, in download
    self.__download_wrapper(self.extract_info)(
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 3333, in wrapper
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1491, in extract_info
    return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1520, in wrapper
    self.report_error(str(e), e.format_traceback())
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 1012, in report_error
    self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
  File "/Users/stanislaw/Library/Caches/pypoetry/virtualenvs/yt-to-anki-f_Xvcl6E-py3.11/lib/python3.11/site-packages/yt_dlp/YoutubeDL.py", line 952, in trouble
    raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: ERROR: [youtube] FVGS1pgaZQ4: Unable to extract uploader id; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

Error starting addon

Hi,

I was really excited to try this addon when I found it. There are several YT videos I would love to turn into decks. So I installed it on Anki, but whenver I try to open the addon I get an error.

Not sure if this is still mantained. I hope it is! Thanks in any case.

Here's the exception I'm getting:

Error
An error occurred. Please start Anki while holding down the shift key, which will temporarily disable the add-ons you have installed.
If the issue only occurs when add-ons are enabled, please use the Tools > Add-ons menu item to disable some add-ons and restart Anki, repeating until you discover the add-on that is causing the problem.
When you've discovered the add-on that is causing the problem, please report the issue on the add-on support site.
Debug info:
Anki 2.1.35 (84dcaa86) Python 3.8.0 Qt 5.14.2 PyQt 5.14.2
Platform: Windows 10
Flags: frz=True ao=True sv=2
Add-ons, last update check: 2020-12-06 15:05:05

Caught exception:
Traceback (most recent call last):
  File "C:\Users\user\AppData\Roaming\Anki2\addons21\1633834851\yt_main.py", line 104, in launch
    screen.setup_ui()
  File "C:\Users\user\AppData\Roaming\Anki2\addons21\1633834851\yt_main.py", line 21, in setup_ui
    fields = self.get_fields()
  File "C:\Users\user\AppData\Roaming\Anki2\addons21\1633834851\yt_main.py", line 40, in get_fields
    pyjson = json.loads(modeljson)
  File "json\__init__.py", line 357, in loads
  File "json\decoder.py", line 337, in decode
  File "json\decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)