feedox / alt-gpt Goto Github PK

Check out the OpenAI client-side connector here as a reference.
Should be relatively easy to add a simple HTTP wrapper for other LLM providers, including HF.
Though, worth checking per each if it supports streaming.

hej love your alt-chatgpt project!👏
How about integrating getitAI's open Plugin Store for easy plugin installs? (https://www.getit.ai/gpt-plugins) Let's chat for 15 mins: https://calendly.com/d/z4j-cb5-h2y/getitai-devs

Follow the instructions by OpenAI here and create a simple manifest that could be used for local development of a plugin and testing it within the Chat page.

We currently use LangchainJS (https://github.com/hwchase17/langchainjs) to load the AI-Plugins and process the response. Since it does not support client-side handling, we have to send the API keys to a node BE, let it process and get a response.
This has a few drawbacks:

1. Playground shouldn't sent apikeys except directly to the LLM provider
2. Agents in Langchain don't have proper control over external (HTTP requests, web pages) that are being used with the AiPlugin, causing overflow of text and tokens being sent to LLMs.
3. Execution time is very slow
4. No streaming

Ideally, we should have a solution where the selected plugins are being executed in client-side, and present the progress in the chat window step-by-step until getting the final result.

currently there's no indication for streaming in progress and no ability to stop and start again