Please add support for LiteLLM to this project about restai HOT 4 CLOSED

Et voila, you can now use LiteLLM :)
If you use an user with admin privileges just add a new LLM using the "LiteLLM" class and specify the parameters you want for this LLM.

In master, published in the next release.

from restai.

We already support any LLM that Ollama supports, you don't need any code to add a LLM. You are able to do it via browser in this case :)
Besides Ollama for local LLMs, any public LLM supported by Llamaindex is also easily supported.

So I believe we are well served regarding LLMs for now :)

from restai.

https://docs.llamaindex.ai/en/stable/examples/llm/litellm/ its veryyy easy to add support for litellm in RestAI.
I will add it in the next release :)

from restai.

@apocas thanks will test later for sure :-)).Problem with ollama only was that in ollama you have to download and then run the heavy model on your system and then you can use its base url in other project like this one which will act as llm source to generate response but in case of litellm we can use any kind of model be it close source like open ai,claud or Gemini or open source model running with public api like groq which is providing mixtral,Gemma,llama or running locally by ourself with ollama so it solves the issue of running multiple types of llm with single api structure which is very convenient and easy to use and that's why i wanted that in this project

from restai.