Giter Club home page Giter Club logo

Comments (17)

artiction avatar artiction commented on June 16, 2024 1

GREAT IT WORKS AFTER 2 DAYS, I should've reported sooner.
At first I did think it was a connection problem but since it worked perfectly 2 days before the bug begins and I haven't been changing connections, I crossed that out. I also thought about bad gateways but if I remember correctly, that should show in the tracebacks so I reported this bug.

Thank you so much! The --cookies worked perfectly. Also to @atnjqt, I was completely aware to that so no worries.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

Hi, you seem to be using an outdated version of the software. chat_replay_downloader is the old name and has since been replaced with chat_downloader (since it is not able to get live stream chats as well.

You can uninstall the old version with

pip uninstall chat-replay-downloader

and then install the new version with

pip install chat-downloader

The output using the new version gives

$ python -m chat_downloader --message_types "text_message paid_message membership_item" https://www.youtube.com/watch?v=aqXFQuxgo5c -o aqXFQuxgo5c1.txt                                                                                                              
[INFO] Site: youtube.com
[ERROR] Members-only content. Join this channel to get access to members-only content like this video, and other exclusive perks.

which is correct.

from chat-downloader.

artiction avatar artiction commented on June 16, 2024

This is come cursed stuff.

python -m chat_downloader --message_types "text_message paid_message membership_item" https://www.youtube.com/watch?v=aqXFQuxgo5c -o aqXFQuxgo5c1.txt
[INFO] Site: youtube.com
Traceback (most recent call last):
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\__main__.py", line 4, in <module>
    main()
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\cli.py", line 210, in main
    chat = downloader.get_chat(**chat_params)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\chat_downloader.py", line 135, in get_chat
    info = self.sessions[site.__name__].get_chat(**params)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1290, in get_chat
    return self.get_chat_by_video_id(match.group('id'), kwargs)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1264, in get_chat_by_video_id
    initial_info = self._get_initial_video_info(video_id)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\lib\site-packages\chat_downloader\sites\youtube.py", line 793, in _get_initial_video_info
    raise ParsingError(
chat_downloader.errors.ParsingError: Unable to parse video data. Please try again.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

Oh that is very strange? It might be an issue with python 3.9, let me check it out.

from chat-downloader.

mikoim avatar mikoim commented on June 16, 2024

@artiction You may be banned from YouTube. Try to fetch the video page with curl. If you see the CAPTCHA page, your IP address has been blocked.

curl 'https://www.youtube.com/watch?v=aqXFQuxgo5c'

Even if your IP address is banned, you can access YouTube as usual through web browsers like Google Chrome.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

I can confirm this is not an error with Python 3.9. Let me add some better error messaging to see what the actual error message is.

Are you able to use tools such as youtube-dl?

Something like this can usually be fixed by specifying a cookies.txt using the --cookies command. See https://github.com/ytdl-org/youtube-dl/#http-error-429-too-many-requests-or-402-payment-required and https://github.com/ytdl-org/youtube-dl/#how-do-i-pass-cookies-to-youtube-dl for more information

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

@artiction Okay, I have improved some of the error messages in this branch.

Can you try run that code (with the -v argument), for us to see what the error actually is?

from chat-downloader.

atnjqt avatar atnjqt commented on June 16, 2024

@artiction I think your issue is that the youtube URL in reference is member only content https://www.youtube.com/watch?v=aqXFQuxgo5c...

I tried to replicate the following in Python (Jupyter notebook):

chat = ChatDownloader().get_chat(url='https://www.youtube.com/watch?v=aqXFQuxgo5c',message_types=["text_message", "paid_message", "membership_item"])

which returns the following error as expected (not sure if there is a way to authenticate):

VideoUnplayable: Members-only content. Join this channel to get access to members-only content like this video, and other exclusive perks.

If you try on another URL with confirmed public chat you should be good to go on python 3.9 I'm guessing... So not really a bug!

An FYI that I found this thread given I am revisiting a dev project from 08/2020 where I used the old chat-replay-downloader ...

The new chat-downloader works really nice but you changed the message_types from chat, superchat, all to "text_message", "paid_message", "membership_item"!

from chat-downloader.

atnjqt avatar atnjqt commented on June 16, 2024

On a different note is there a python flag for --output so I can export chats to json?

I see this in the CLI documentation (you included -o flag in your cmd) but not for the ChatDownloader().get_chat() ... I need this json so I can pd.read_json for ETL of chat data

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-106-472d1877d159> in <module>
----> 1 chat = ChatDownloader().get_chat(url=url,message_types=["text_message", "paid_message", "membership_item"],output=outfile)

TypeError: get_chat() got an unexpected keyword argument 'output'

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

On a different note is there a python flag for --output so I can export chats to json?

I see this in the CLI documentation (you included -o flag in your cmd) but not for the ChatDownloader().get_chat() ... I need this json so I can pd.read_json for ETL of chat data

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-106-472d1877d159> in <module>
----> 1 chat = ChatDownloader().get_chat(url=url,message_types=["text_message", "paid_message", "membership_item"],output=outfile)

TypeError: get_chat() got an unexpected keyword argument 'output'

Do you mind opening another issue for us to discuss there? So that the current thread doesn't get too long.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

GREAT IT WORKS AFTER 2 DAYS, I should've reported sooner.
At first I did think it was a connection problem but since it worked perfectly 2 days before the bug begins and I haven't been changing connections, I crossed that out. I also thought about bad gateways but if I remember correctly, that should show in the tracebacks so I reported this bug.

Thank you so much! The --cookies worked perfectly. Also to @atnjqt, I was completely aware to that so no worries.

Great! No worries :) That's the main reason --cookies was implemented.

@artiction If possible, could you still run this code with the same arguments using this branch: https://github.com/xenova/chat-downloader/tree/errors (soon to be added to the main branch) ... Just so I can actually see what the debugger says. Just so that whenever someone else has the same issue they can be told to use --cookies. Unfortunately I have never had an issue like this, so I was not able to code for this happening.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

Okay I have added the debugging functionality to the latest version which you can install by running:

pip install chat-downloader --upgrade

After doing so, can you run the original command so that we can see what the errors were?

chat_downloader -v --message_types "text_message paid_message membership_item" https://www.youtube.com/watch?v=aqXFQuxgo5c

I have reopened the issue, but I will close it after you respond :)

Thanks!

from chat-downloader.

artiction avatar artiction commented on June 16, 2024 
File "c:\users\x\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\x\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\x\AppData\Local\Programs\Python\Python39\Scripts\chat_downloader.exe\__main__.py", line 7, in <module>
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\cli.py", line 170, in main
    run(**args.__dict__)
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\chat_downloader.py", line 273, in run
    chat = downloader.get_chat(**chat_params)
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\chat_downloader.py", line 166, in get_chat
    info = self.sessions[site.__name__].get_chat(**params)
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1347, in get_chat
    return self.get_chat_by_video_id(match.group('id'), kwargs)
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1321, in get_chat_by_video_id
    initial_info = self._get_initial_video_info(video_id)
  File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 871, in _get_initial_video_info
    raise ParsingError(
chat_downloader.errors.ParsingError: Unable to parse video data. Please try again.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

@artiction Are you sure this is the full response? Did you run it in verbose mode? (-v)

The latest updates to the code (which you have to manually download/update, either using pip or git) include debugging information for what was parsed:

chat-downloader/chat_downloader/sites/youtube.py

Lines 862 to 872 in 623b699

def _get_initial_video_info(self, video_id):
""" Get initial YouTube video information. """
original_url = self._YT_VIDEO_TEMPLATE.format(video_id)
html, yt_initial_data = self._get_initial_info(original_url)
player_response = re.search(self._YT_INITIAL_PLAYER_RESPONSE_RE, html)
if not yt_initial_data:
log('debug', html)
raise ParsingError(
'Unable to parse video data. Please try again.')

On line 870.

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

@artiction Sorry for bothering again, but I have just updated the code to include the error messaging changes I mentioned above (https://github.com/xenova/chat-downloader/releases/tag/v0.0.6)

Could you perhaps update, then run the command again and post the error message?

pip install chat-downloader --upgrade
chat_downloader -v --message_types "text_message paid_message membership_item" https://www.youtube.com/watch?v=aqXFQuxgo5c

It would be great to be able to parse the error and display relevant messages to the user. Since I have never had the error, I haven't been able to do this.

Thanks!

from chat-downloader.

artiction avatar artiction commented on June 16, 2024

Traceback (most recent call last):

File "c:\users\x\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\x\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\x\AppData\Local\Programs\Python\Python39\Scripts\chat_downloader.exe_main.py", line 7, in
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\cli.py", line 170, in main
run(**args.dict)
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\chat_downloader.py", line 273, in run
chat = downloader.get_chat(**chat_params)
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\chat_downloader.py", line 166, in get_chat
info = self.sessions[site.name].get_chat(**params)
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1385, in get_chat
return self.get_chat_by_video_id(match.group('id'), kwargs)
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 1359, in get_chat_by_video_id
initial_info = self._get_initial_video_info(video_id)
File "c:\users\x\appdata\local\programs\python\python39\lib\site-packages\chat_downloader\sites\youtube.py", line 905, in _get_initial_video_info
raise ParsingError(
chat_downloader.errors.ParsingError: Unable to parse initial video data. <title>YouTube</title><script src="https://www.google.com/recaptcha/api.js?trustedtypes=true&hl=en" async defer nonce="3ci5Sy5DF1zXl5zRJ9aekQ"></script><style nonce="3ci5Sy5DF1zXl5zRJ9aekQ">html{height:100%;margin:0;padding:0;border:0;font-size:100%;background:transparent}body{margin:0;padding:0;border:0;font:12px "YouTube Noto",Roboto,arial,sans-serif;background:#f1f1f1;height:100%}#captcha-page{text-align:center;height:100%;white-space:nowrap;word-break:normal}.g-recaptcha{display:inline-block}#captcha-page-content{vertical-align:middle;display:inline-block}#captcha-page-vertical-align{height:100%;display:inline-block;vertical-align:middle}p{margin:auto;color:#333;font-size:16px;white-space:normal;text-shadow:0 0 0 transparent,0 1px 1px #fff}</style>

Sorry for the interruption. We have been receiving a large volume of requests from your network.

To continue with your YouTube experience, please fill out the form below.


<style nonce="3ci5Sy5DF1zXl5zRJ9aekQ">#yt-masthead{margin:15px auto;width:440px;margin-top:25px}#logo-container{margin-right:5px;float:left;cursor:pointer;text-decoration:none}.logo{background:no-repeat url("//www.gstatic.com/youtube/img/branding/youtubelogo/1x/youtubelogo_30.png");width:125px;height:30px;cursor:pointer;display:inline-block}#masthead-search{display:flex;margin-top:3px;max-width:650px;overflow:hidden;padding:0;position:relative}.search-button{border-left:0;-moz-border-radius-topleft:0;border-top-left-radius:0;-moz-border-radius-bottomleft:0;border-bottom-left-radius:0;float:right;height:29px;padding:0;border:solid 1px transparent;border-color:#d3d3d3;background:#f8f8f8;color:#333;cursor:pointer}.search-button:hover{border-color:#c6c6c6;background:#f0f0f0;box-shadow:0 1px 0 rgba(0,0,0,0.10)}.search-button-content{border:none;display:block;opacity:.6;padding:0;text-indent:-10000px;background:no-repeat url(//www.gstatic.com/youtube/src/web/htdocs/img/search.png);background-size:auto;width:15px;height:15px;box-shadow:none;margin:0 25px}#masthead-search-terms-border{flex:1 1 auto;border:1px solid #ccc;box-shadow:inset 0 1px 2px #eee;background-color:#fff;font-size:14px;height:29px;line-height:30px;margin:0 0 2px;overflow:hidden;position:relative;-moz-box-sizing:border-box;box-sizing:border-box;-webkit-transition:border-color .2s ease;transition:border-color .2s ease}#masthead-search-terms{background:transparent;border:0;font-size:16px;height:100%;left:0;margin:0;outline:none;padding:2px 6px;position:absolute;width:100%;-moz-box-sizing:border-box;box-sizing:border-box}</style>
ID
Search

from chat-downloader.

xenova avatar xenova commented on June 16, 2024

Perfect!!! Thank you so much - this is invaluable information! 🔥

Unable to parse initial video data. <title>YouTube</title><script src="https://www.google.com/recaptcha/api.js?trustedtypes=true&hl=en" async defer nonce="3ci5Sy5DF1zXl5zRJ9aekQ"></script><style nonce="3ci5Sy5DF1zXl5zRJ9aekQ">html{height:100%;margin:0;padding:0;border:0;font-size:100%;background:transparent}body{margin:0;padding:0;border:0;font:12px "YouTube Noto",Roboto,arial,sans-serif;background:#f1f1f1;height:100%}#captcha-page{text-align:center;height:100%;white-space:nowrap;word-break:normal}.g-recaptcha{display:inline-block}#captcha-page-content{vertical-align:middle;display:inline-block}#captcha-page-vertical-align{height:100%;display:inline-block;vertical-align:middle}p{margin:auto;color:#333;font-size:16px;white-space:normal;text-shadow:0 0 0 transparent,0 1px 1px #fff}</style>

Sorry for the interruption. We have been receiving a large volume of requests from your network.

To continue with your YouTube experience, please fill out the form below.


<style nonce="3ci5Sy5DF1zXl5zRJ9aekQ">#yt-masthead{margin:15px auto;width:440px;margin-top:25px}#logo-container{margin-right:5px;float:left;cursor:pointer;text-decoration:none}.logo{background:no-repeat url("//www.gstatic.com/youtube/img/branding/youtubelogo/1x/youtubelogo_30.png");width:125px;height:30px;cursor:pointer;display:inline-block}#masthead-search{display:flex;margin-top:3px;max-width:650px;overflow:hidden;padding:0;position:relative}.search-button{border-left:0;-moz-border-radius-topleft:0;border-top-left-radius:0;-moz-border-radius-bottomleft:0;border-bottom-left-radius:0;float:right;height:29px;padding:0;border:solid 1px transparent;border-color:#d3d3d3;background:#f8f8f8;color:#333;cursor:pointer}.search-button:hover{border-color:#c6c6c6;background:#f0f0f0;box-shadow:0 1px 0 rgba(0,0,0,0.10)}.search-button-content{border:none;display:block;opacity:.6;padding:0;text-indent:-10000px;background:no-repeat url(//www.gstatic.com/youtube/src/web/htdocs/img/search.png);background-size:auto;width:15px;height:15px;box-shadow:none;margin:0 25px}#masthead-search-terms-border{flex:1 1 auto;border:1px solid #ccc;box-shadow:inset 0 1px 2px #eee;background-color:#fff;font-size:14px;height:29px;line-height:30px;margin:0 0 2px;overflow:hidden;position:relative;-moz-box-sizing:border-box;box-sizing:border-box;-webkit-transition:border-color .2s ease;transition:border-color .2s ease}#masthead-search-terms{background:transparent;border:0;font-size:16px;height:100%;left:0;margin:0;outline:none;padding:2px 6px;position:absolute;width:100%;-moz-box-sizing:border-box;box-sizing:border-box}</style>
ID
Search

I don't suppose there is any text below "ID Search"?

from chat-downloader.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.