paweld2 / captions Goto Github PK

This repository contains captions for videos on 3blue1brown. The files of the form year/video-id/language/sentence_translations.json are used to automatically create subtitles, and the goal is to use them to create automatic dubbing as well. As a proof of concept for the aim here, on this video, click the gear icon, and change the audio track to "Korean", or take a look a this one, and try the audio tracks in French, Greman, Hindi or Spanish.

Most of this repo was formed with a script that first used Whisper to transcribe video narration into English with punctuation and then used a translation API to translate that script sentence-by-sentence, recording the timings. Machine translation is often far from perfect, so I'll only feel comfortable with the final results here if those translations have been looked over by a native speaker.

We're figuring out the best way to incorporate community contributions. If you'd like to help out, it might be best to hold off until a proper system is in place, and note your interest in this Google form, or to join this discord.