Giter Club home page Giter Club logo

dinoosauro / tiktok-to-ytdlp Goto Github PK

View Code? Open in Web Editor NEW
129.0 4.0 13.0 244 KB

Fetch all the liked videos, videos from an user, videos with a specific sound etc. from TikTok, and creates a script to download them with yt-dlp

License: MIT License

JavaScript 47.50% HTML 0.84% Svelte 39.00% CSS 6.23% TypeScript 6.42%
tiktok tiktok-api tiktok-downloader tiktok-likes tiktok-scraper downloader tiktok-sounds tiktok-videos-download tiktokapi video

tiktok-to-ytdlp's Introduction

tiktok-to-ytdlp

Fetch all the liked videos, videos from an user, videos with a specific sound etc. from TikTok, and creates a script to download them with yt-dlp

Instructions

From the extension

Download the extension by following the instructions you can find in the extension README. Then, change the conversion options (if you want to), and click on the button to start the conversion. The extesion will take care of everything else.

From the console

Open the TikTok webpage of the user/sound/etc. you want all the videos downloaded. Press Ctrl (or Cmd if you are on a Mac) + Shift + I to open the Developer Tools. Go into the Console tab on the top (if you don't see it, click on the >>) and paste the content you can find in the script.js file. If you prefer a minified version, you can find that here. Press enter.

Next steps

The webpage will automatically scroll until no other items are found. Then, a file called "TikTokLinks.txt" will be downloaded. You now can download the videos with yt-dlp. An example script is:

yt-dlp -a TikTokLinks.txt -o "TikTok/%(uploader)s/%(title)s - %(id)s.%(ext)s"

Note that the most important part in this script is -a TikTokLinks.txt: by writing this, yt-dlp will download all the videos that are in the txt file. You can then add all the arguments you prefer to yt-dlp

Ask for intermediate files

If you need to download a really long list of TikTok, you may want to start downloading them while the page continues scrolling. To do that, you can download an intermediate file, that will contain all of the links up to the point the page has scrolled. To do that, write in the console requestTxtNow(), press enter and a TikTokLinks.txt file will be downloaded.

By default, the links in the first intermediate files will be deleted from the final (or the second/third etc. file if you want to download more intermediate files) file, so that you won't download any duplicates. You can change that by putting the delete_from_next_txt option to false.

Script options

You can edit the values of the first five lines of the script to change some useful settings:

  • scrolling_min_time & scrolling_max_time: change the thread sleeps between a scroll and the next one.
  • min_views: don't add a video to the text file if it has fewer than x views.
  • delete_from_next_txt: put this to false if you want that the final txt files has all the links, even the ones you've already downloaded with intermediate files.
  • output_name_type: choose the format for the output file name. Valid inputs are:
    • A generic string: the file name will be exactly that
    • 0 (as integer): try fetching the title from some data tags
    • 1 (as integer): the webpage title will be used
    • 2 (as integer; default): the first title on the page (h1 HTML elmenet) will be used
    • Any other value: "TikTokLinks.txt" will be used as a file name
  • adapt_text_output: replace Windows unsafe characters for the output file name.
  • allow_images: save also TikTok photos (if disabled, only videos will be fetched).

Advanced script options

These really useful options permit to change how the script fetches data, and what it should do when errors are found.

  • get_array_after_scroll: change this if you want to fetch all the TikTok videos when the script has finished scrolling, and not after every single scroll. Keep in mind that it seems that TikTok doesn't unappend the previous videos, so this option shouldn't be relevant, but this might change in the future.
  • get_link_by_filter: use the new method of getting all the links in a video container, and then look for the one that contains the video URL structure. If false, the "old" method of using data attributes will be used (even if I couldn't find more tags, I can't garantee that this old method has all of them).
  • check_nullish_link: check if the link is nullish, and, since it would be useless to append it, continue with the next link. It's reccomended to leave this set to true.
  • log_link_error: write to the console if a link is nullish.
  • maximum_downloads: if the number of fetched items is above this number, the fetching process will be stopped
    • The get_array_after_scroll variable must be set to true
    • If more items than this number can be fetched without continuing scrolling, they'll also be added. Therefore, the output number might be greater than this variable.

Warning:

This script is licensed under the MIT license.

Even if basically the only thing this script does is automatically scrolling the webpage, so there's not that big of a risk, I don't claim any responsibilties at all for the usage of this script and the eventual consequences.

tiktok-to-ytdlp's People

Contributors

cassian-andor avatar dinoosauro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

tiktok-to-ytdlp's Issues

Empty file

Hi!
The script is currently downloading empty files. Maybe TikTok sources changed?
It seems to scroll to the end just fine.

Tested on Chrome and Safari.

limit the number of fetched files

Hi, I wonder if this script can limit the number of fetched files, if we do not want all the videos. Thank you.
Also, I found that this can not fetch links from tiktok.com/search/?

Docs need updating

Thanks for this awesome tool.

Two things that the docs should be updated to include:

(1) When you try to paste the script into the console, you get the error that you have to type allow pasting first. This should probably be in your instructions:

image

(2) However once the scraper completes, it also looks like the download failed:

image

These errors can be ignored, however -- the file containing URLs should have been written to the Downloads folder.

i keep getting an error

i keep getting an error saying "the JSON object must be str, bytes or bytearray, not NoneType"

Crediting you for a userscript using your code

Hi Dinoosauro, sorry for adding this as an issue. I've written an Userscript that uses this code as a base to add a "Get list of tiktok links" button to Tiktok, and I wanted to know the best way to credit your work. The metadata links to this Github repo link, and mentions you both on the @description and on the @author tag. I've also made to sure to add credits on the Readme: https://github.com/lihuelworks/tiktok-to-ytdlp-userscript

Let me know if there is a better way to credit your work.

Can we migrate this script to node script

Hi there,
Thank for your awsome work.

This script only run in console window, so that we must open browser to run script.
Now Can we migrate this to node js framework. I mean we just input the tiktok user id, node js will convert this script then run all thing to fetch videos then download it.
That's my idea.

Thank again for your script

error after scrolling

hey! thanks for the great tool--this had been working for me, but am now hitting the following error, and not sure how to adjust the script:

Uncaught TypeError: Cannot read properties of null (reading 'querySelector') at addArray (<anonymous>:44:305) at ytDlpScript (<anonymous>:52:5) at <anonymous>:26:25

for ref I ran script.js as is, just with get_array_after_scroll set to true -- scrolling works fine and then nothing gets written out. thanks in advance!

Pass download file name (or guess from URL?)

Hi. This is a great script thank you. Works fantastic on Safari. However for automation, I find it cumbersome to rename the file every time.
I was thinking of making a "ViolentMonkey" script that only triggers on certain profiles (ones to download), includes your script, and then auto starts scrolling (essentially removing the need to go to console). I am not sure if the entry point would be loadWebpage then, or none because it's already called in the script definition.

In order to generate different txt files to pass to yt-dlp, because they all have the same filename (TikTokLinks.txt), you have to rename each, every time. And if you have multiple channels then it gets a bit tiring.

Would it be possible to maybe pass the channel name to your script, so that the resulting filename would be that? Or maybe for the script to auto-fetch the name? In a way that we don't need to modify the script at all, and we can just include it from other scripts (remotely) so that it's always up-to-date and whatnot.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.