In this repo you will find resources, demos, recipes... to work with LLMs on OpenShift with OpenShift Data Science or Open Data Hub.

Two different Inference Servers deployment instructions are available:

• Caikit-TGIS-Serving (external): How to deploy the Caikit-TGIS-Serving stack, from OpenDataHub.
• vLLM: How to deploy vLLM, the "Easy, fast, and cheap LLM serving for everyone".
• Hugging Face Text Generation Inference: How to deploy the Text Generation Inference server from Hugging Face.

• vLLM Serving Runtime: How to deploy a vLLM runtime in the Single-Model Serving stack of Open Data Hub or OpenShift AI.

Deployments of different databases that can be used as a Vector Store are available:

• PostgreSQL+pgvector: Full recipe to create an instance of PostgreSQL with the pgvector extension, making it usable as a vector store.
• Redis: Full recipe to deploy Redis, create a Cluster and a suitable Database for a Vector Store.

• Caikit: Basic example demonstrating how to work with Caikit+TGIS for LLM serving.
• Langchain examples: Various notebooks demonstrating how to work with Langchain. Examples include both HFTGI and Caikit+TGIS Serving.
• Langflow examples: Various examples demonstrating how to work with Langflow.
• UI examples: Various examples on how to create and deploy a UI to interact with your LLM.