Streaming responses about chatgpt HOT 36 CLOSED

Example code: #5 (comment)

from chatgpt.

Opened #23

from chatgpt.

Happy to help, whether that means picking up where you left off or figuring it out from scratch...

A pull request would be much appreciated.

Perhaps there could be another parameter to get_chat_response to specify the return type: text or stream, with text as default

from chatgpt.

Ok, I'll try to take a look tomorrow. LMK if you find anything out about how it's done in the browser: I haven't done any research yet.

from chatgpt.

I implemented streaming response, but I don't know how to output it on the console.

from chatgpt.

I implemented streaming response, but I don't know how to output it on the console.

I solved the problem, but the method was ugly.

from chatgpt.

@A-kirami I have written a rough implementation of text streaming. Can you check if it works as intended (It works for me but need verification)

get_chat_response(prompt, output="stream") should yield each message.

if __name__ == "__main__":
    with open("config.json", "r") as f:
        config = json.load(f)
    chatbot = Chatbot(config)
    if 'session_token' in config:
        chatbot.refresh_session()
    while True:
        prompt = input("You: ")
        for message in chatbot.get_chat_response(prompt, output="stream"):
            print(message)

This should print out the message as it streams. I'm not sure how to deal with the console so that it is overwritten as more data comes in.

from chatgpt.

@A-kirami I have written a rough implementation of text streaming. Can you check if it works as intended (It works for me but need verification)

get_chat_response(prompt, output="stream") should yield each message.

if __name__ == "__main__":
    with open("config.json", "r") as f:
        config = json.load(f)
    chatbot = Chatbot(config)
    if 'session_token' in config:
        chatbot.refresh_session()
    while True:
        prompt = input("You: ")
        for message in chatbot.get_chat_response(prompt, output="stream"):
            print(message)

This should print out the message as it streams. I'm not sure how to deal with the console so that it is overwritten as more data comes in.

It works normally. If you need to print it on the console, you can refer to the implementation here.

https://github.com/A-kirami/ChatGPT/blob/main/src/revChatGPT/__main__.py#L69-L81

from chatgpt.

@acheong08

Like this:

from chatgpt.

I am terrible at this. It keeps erasing the lines in my implementation

from chatgpt.

I'll copy your code over and modify it

from chatgpt.

This doesn't work in all terminals:

Detected that the lines of output has been exceeded the height of terminal windows, which                     caused the former output remained and keep adding new lines.
检测到输出过程中, 输出行数曾大于命令行窗口行数, 这会导致输出清除不完整, 而使输出不停增长。请注意控制输出行数

from chatgpt.

sys.stdout.write("\033[F\033[K")

This doesn't work properly in all terminals either for some reason. My hair is falling off lol

from chatgpt.

The error seems to be that this deletes the number of lines it is aware of but in some terminals, line count is counted differently

from chatgpt.

The approach I was attempting (but haven't perfected yet) is to only write a line once it is has completely loaded, rather than deleting lines after they're written. Would that work here?

from chatgpt.

That is what I'm attempting to do. I write the whole message and then when a new one comes in, I try to delete the previous message. It's not clearing the right number of lines though.

from chatgpt.

I'm suggesting you not try to delete the previous message (as I did with the please wait prompt), but rather keep track of how many lines you've written, and only write each new line.

from chatgpt.

keep track of how many lines you've written

Tried that. An issue with this is that some environments treat lines differently. For example, my current omz shell treats each new line in the terminal as a new line (which changes depending on window size). However, Python can only see \n as new line

from chatgpt.

I tried counting the characters and saving them, then deleting that number of characters when a new message comes in. However, \b isn't supported in some terminals either.

from chatgpt.

Not even ChatGPT knows the solution to this...

from chatgpt.

I think streaming is more suited for GUI applications. Terminals don't like overwrites

from chatgpt.

Do you have a pointer to a commit that has streaming working but doesn't have the terminal stuff solved yet? I can restart my efforts from there.

from chatgpt.

Streaming working on latest commit

from chatgpt.

Of main or async-dev?

from chatgpt.

main

from chatgpt.

On that commit I'm getting

  File "/Users/sleibrand/src/ChatGPT/src/revChatGPT/__main__.py", line 70, in <module>
    print("Chatbot:", response['message'])
TypeError: 'generator' object is not subscriptable

from chatgpt.

It returns a string. I didn't update the docs yet. You need to call with output="stream"

from chatgpt.

I've been running it with python3 -m revChatGPT - is there another command-line you've been using?

from chatgpt.

I've been running it with python3 -m revChatGPT - is there another command-line you've been using?

It can only be implemented custom right now.

from chatgpt.

https://github.com/acheong08/ChatGPT/compare/main...scottleibrand:ChatGPT:async?expand=1

~/src/ChatGPT $ python3 -m revChatGPT

    ChatGPT - A command-line interface to OpenAI's ChatGPT (https://chat.openai.com/chat)
    Repo: github.com/acheong08/ChatGPT

Type '!help' to show commands
Press enter twice to submit your question.

You: test

...
I'm sorry, but I'm not sure what you're looking for. Could you please provide
more information or context? I'm a large language model trained by OpenAI, and
I'm here to help with any questions you may have.




You:

Cleaning up, then will PR.

from chatgpt.

@scottleibrand

Chatbot: 
Something went wrong!
name 'formatted_parts' is not defined

from chatgpt.

I ought to have some better way to detect auth errors

from chatgpt.

Yeah, that has tripped me up a few times too. Can we store more persistent credentials to allow it to log in again?

from chatgpt.

The thing with the next auth tokens is that it needs to be refreshed every hour via !refresh. It stays valid that way. However, if you don't refresh it, it expires.

from chatgpt.

I haven't figured out their hashing system on the login page. It hashes the password client side

from chatgpt.

I'll open an issue for this

from chatgpt.