https://github.com/labring/FastGPT/
FastGPT is a knowledge-based question answering system built on the LLM. It offers out-of-the-box data processing and model invocation capabilities. Moreover, it allows for workflow orchestration through Flow visualization, thereby enabling complex question and answer scenarios!

Maybe provide an another list by activity of project, there are some projects haven't merge new code for years.

thanks for adding this list. I propose to add some of the following links under large-model-serving. In particular, some of the repos are not for decoder models, but more for sentence transformers or clip.

Embeddings from OpenAI clip.

Jina
https://github.com/jina-ai/clip-as-service (Apache)

Text-embeddings:

My own project: infinity ( just add it if you like it)
https://github.com/michaelfeil/infinity (MIT)
https://github.com/huggingface/text-embeddings-inference (no opensource Licence)

LLM Inference as a service:

Huggingface/ TGI
https://github.com/huggingface/text-generation-inference (no opensource licence after 1.0)
Nvidia-TensorRTLLM (Apache2.0)

• torchserve
  • https://github.com/pytorch/serve
• neuropod
  • https://github.com/uber/neuropod
  • A uniform interface to run deep learning models from multiple frameworks
  • though it is not actively maintain now

There are some great closed-sourced llmops tools e.g. pinecone, humanloop.

I think we should also add them to the list.

ppull request

https://fedml.ai

Found your compilation list very useful.
Recently have been using this helpful OpenSource repo to build ChatGPT like bots; it requires very less coding and could help people.

Embedchain:
https://github.com/embedchain/embedchain