How to Stop Generating through Curl + Sockets about h2ogpt HOT 3 OPEN

Hi, the stop in UI works by stopping the async call, but the thread doing the generation still continues. It's challenging in general to do an actual stop. This is not possible for vLLM or OpenAI or other platforms either. Same with GGUF, AWQ, etc. No way to stop them.

For torch itself, I can hack something like this: #298 . But it's not general.

However, I can potentially apply for the streaming case if used and use for all (or most) model types.

from h2ogpt.

@pseudotensor please guide

from h2ogpt.

However, I can potentially apply for the streaming case if used and use for all (or most) model types.

That would be wonderful and very helpful!

Where can I find documentation of CLI arguments? I'm interested to know what --early_stopping do

from h2ogpt.