A Kubernetes operator for managing Red Hat OpenShift Lightspeed.
You'll need an OpenShift cluster to run against.
Important
The Operator only supports OpenAI and BAM as large language model (LLM) providers.
Note: Your controller will automatically use the current context from your kubeconfig file (i.e. whatever cluster oc cluster-info shows).
- Deploy the controller to the cluster:
make deploy- Create a secret containing the API Key for BAM or OpenAI. The key for API key is
apitoken.
Tip
OpenAPI example
apiVersion: v1
data:
apitoken: <base64 encoded API Key>
kind: Secret
metadata:
name: openai-api-keys
namespace: openshift-lightspeed
type: OpaqueTip
BAM example
apiVersion: v1
data:
apitoken: <base64 encoded API Key>
kind: Secret
metadata:
name: bam-api-keys
namespace: openshift-lightspeed
type: Opaque- Create an
OLSConfigcustom resource
apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
name: cluster
namespace: openshift-lightspeed
spec:
llm:
providers:
- credentialsSecretRef:
name: openai-api-keys
models:
- name: gpt-3.5-turbo
name: openai
url: https://api.openai.com/v1
- credentialsSecretRef:
name: bam-api-keys
models:
- name: ibm/granite-13b-chat-v2
name: bam
url: https://bam-api.res.ibm.com
ols:
conversationCache:
redis:
maxMemory: 2000mb
maxMemoryPolicy: allkeys-lru
type: redis
defaultModel: ibm/granite-13b-chat-v2
defaultProvider: bam
enableDeveloperUI: false
logLevel: INFO
deployment:
replicas: 1- The Operator will reconcile the CustomResource (CR) and create all the necessary resources for launching the
Red Hat OpenShift Lightspeedapplication server.
To delete the CRDs from the cluster:
make uninstallUnDeploy the controller from the cluster:
make undeploy- Create a namespace
openshift-lightspeed
oc create namespace openshift-lightspeed- Install the CRDs into the cluster:
make install- Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):
make run-
Create a secret containing the API Key for BAM or OpenAI. The key for API key is
apitoken. -
Create an
OLSConfigcustom resource -
The Operator will reconcile the CustomResource (CR) and create all the necessary resources for launching the
Red Hat OpenShift Lightspeedapplication server.
➜ oc get configmaps -n openshift-lightspeed
NAME DATA AGE
kube-root-ca.crt 1 4h9m
olsconfig 1 4h9m
openshift-service-ca.crt 1 4h9m
➜ oc get services -n openshift-lightspeed
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
lightspeed-app-server ClusterIP 172.30.8.150 <none> 8080/TCP 22m
lightspeed-operator-controller-manager-metrics-service ClusterIP 172.30.35.253 <none> 8443/TCP 2d15h
➜ oc get deployments -n openshift-lightspeed
NAME READY UP-TO-DATE AVAILABLE AGE
lightspeed-app-server 1/1 1 1 23m
lightspeed-operator-controller-manager 1/1 1 1 2d15h
➜ oc get pods -n openshift-lightspeed
NAME READY STATUS RESTARTS AGE
lightspeed-app-server-77bd6d666c-4ct7v 1/1 Running 0 23m
lightspeed-operator-controller-manager-6759d8c66d-bfnzs 2/2 Running 0 2d15h
➜ oc logs ols-app-server-5c9765967d-vvwnh -n openshift-lightspeed
2024-02-02 12:00:06,982 [ols.app.main:main.py:29] INFO: Embedded Gradio UI is disabled. To enable set enable_dev_ui: true in the dev section of the configuration file
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)If you have updated the API definitions, you must update the CRD manifests with the following command
make manifestsNOTE: Run make --help for more information on all potential make targets
You'll need the following tools to develop the Operator:
- OperatorSDK, version 1.33
- git
- go, version 1.21
- docker, version 17.03+.
- oc or oc and access to an OpenShift cluster of a compatible version.
- golangci-lint, version v1.54.2
![openshift-merge-bot[bot] avatar](https://avatars.githubusercontent.com/in/412865?v=4 "openshift-merge-bot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent