Yatai (ๅฑๅฐ, food cart) lets you deploy, operate and scale Machine Learning services on Kubernetes.
It supports deploying any ML models via BentoML: the unified model serving framework.
๐ Join our Slack community today!
โจ Looking for the fastest way to give Yatai a try? Check out BentoML Cloud to get started today.
๐ฑ Made for BentoML, deploy at scale
- Scale BentoML to its full potential on a distributed system, optimized for cost saving and performance.
- Manage deployment lifecycle to deploy, update, or rollback via API or Web UI.
- Centralized registry providing the foundation for CI/CD via artifact management APIs, labeling, and WebHooks for custom integration.
๐ Cloud native & DevOps friendly
- Kubernetes-native workflow via BentoDeployment CRD (Custom Resource Definition), which can easily fit into an existing GitOps workflow.
- Native integration with Grafana stack for observability.
- Support for traffic control with Istio.
- Compatible with all major cloud platforms (AWS, Azure, and GCP).
- ๐ Documentation - Overview of the Yatai docs and related resources
- โ๏ธ Installation - Hands-on instruction on how to install Yatai for production use
- ๐ Join Community Slack - Get help from our community and maintainers
Let's try out Yatai locally in a minikube cluster!
- Install latest minikube: https://minikube.sigs.k8s.io/docs/start/
- Install latest Helm: https://helm.sh/docs/intro/install/
- Start a minikube Kubernetes cluster:
minikube start --cpus 4 --memory 4096
, if you are using macOS, you should use hyperkit driver to prevent the macOS docker desktop network limitation - Check that minikube cluster status is "running":
minikube status
- Make sure your
kubectl
is configured withminikube
context:kubectl config current-context
- Enable ingress controller:
minikube addons enable ingress
Install Yatai with the following script:
bash <(curl -s "https://raw.githubusercontent.com/bentoml/yatai/main/scripts/quick-install-yatai.sh")
This script will install Yatai along with its dependencies (PostgreSQL and MinIO) on your minikube cluster.
Note that this installation script is made for development and testing use only. For production deployment, check out the Installation Guide.
To access Yatai web UI, run the following command and keep the terminal open:
kubectl --namespace yatai-system port-forward svc/yatai 8080:80
In a separate terminal, run:
YATAI_INITIALIZATION_TOKEN=$(kubectl get secret yatai-env --namespace yatai-system -o jsonpath="{.data.YATAI_INITIALIZATION_TOKEN}" | base64 --decode)
echo "Open in browser: http://127.0.0.1:8080/setup?token=$YATAI_INITIALIZATION_TOKEN"
Open the URL printed above from your browser to finish admin account setup.
First, get an API token and login to the BentoML CLI:
-
Keep the
kubectl port-forward
command in the step above running -
Go to Yatai's API tokens page: http://127.0.0.1:8080/api_tokens
-
Create a new API token from the UI, making sure to assign "API" access under "Scopes"
-
Copy the login command upon token creation and run as a shell command, e.g.:
bentoml yatai login --api-token {YOUR_TOKEN} --endpoint http://127.0.0.1:8080
If you don't already have a Bento built, run the following commands from the BentoML Quickstart Project to build a sample Bento:
git clone https://github.com/bentoml/bentoml.git && cd ./examples/quickstart
pip install -r ./requirements.txt
python train.py
bentoml build
Push your newly built Bento to Yatai:
bentoml push iris_classifier:latest
Now you can view and manage models and bentos from the web UI:
Yatai's image builder feature comes as a separate component, you can install it via the following script:
bash <(curl -s "https://raw.githubusercontent.com/bentoml/yatai-image-builder/main/scripts/quick-install-yatai-image-builder.sh")
This will install the BentoRequest
CRD(Custom Resource Definition) and Bento
CRD
in your cluster. Similarly, this script is made for development and testing purposes only.
Yatai's Deployment feature comes as a separate component, you can install it via the following script:
bash <(curl -s "https://raw.githubusercontent.com/bentoml/yatai-deployment/main/scripts/quick-install-yatai-deployment.sh")
This will install the BentoDeployment
CRD(Custom Resource Definition)
in your cluster and enable the deployment UI on Yatai. Similarly, this script is made for development and testing purposes only.
Once the yatai-deployment
component was installed, Bentos pushed to Yatai can be deployed to your
Kubernetes cluster and exposed via a Service endpoint.
A Bento Deployment can be created either via Web UI or via a Kubernetes CRD config:
- Go to the deployments page: http://127.0.0.1:8080/deployments
- Click
Create
button and follow the instructions on the UI
Define your Bento deployment in a my_deployment.yaml
file:
apiVersion: resources.yatai.ai/v1alpha1
kind: BentoRequest
metadata:
name: iris-classifier
namespace: yatai
spec:
bentoTag: iris_classifier:3oevmqfvnkvwvuqj # check the tag by `bentoml list iris_classifier`
---
apiVersion: serving.yatai.ai/v2alpha1
kind: BentoDeployment
metadata:
name: my-bento-deployment
namespace: yatai
spec:
bento: iris-classifier
ingress:
enabled: true
resources:
limits:
cpu: "500m"
memory: "512Mi"
requests:
cpu: "250m"
memory: "128Mi"
autoscaling:
maxReplicas: 10
minReplicas: 2
runners:
- name: iris_clf
resources:
limits:
cpu: "1000m"
memory: "1Gi"
requests:
cpu: "500m"
memory: "512Mi"
autoscaling:
maxReplicas: 4
minReplicas: 1
Apply the deployment to your minikube cluster:
kubectl apply -f my_deployment.yaml
Now you can see the deployment process from the Yatai Web UI and find the endpoint URL for accessing the deployed Bento.
- To report a bug or suggest a feature request, use GitHub Issues.
- For other discussions, use GitHub Discussions under the BentoML repo
- To receive release announcements and get support, join us on Slack.
There are many ways to contribute to the project:
- If you have any feedback on the project, share it with the community in GitHub Discussions under the BentoML repo.
- Report issues you're facing and "Thumbs up" on issues and feature requests that are relevant to you.
- Investigate bugs and review other developers' pull requests.
- Contributing code or documentation to the project by submitting a GitHub pull request. See the development guide.
Yatai collects usage data that helps our team to improve the product.
Only Yatai's internal API calls are being reported. We strip out as much potentially
sensitive information as possible, and we will never collect user code, model data, model names, or stack traces.
Here's the code for usage tracking.
You can opt-out of usage by configuring the helm chart option doNotTrack
to
true
.
doNotTrack: false
Or by setting the YATAI_DONOT_TRACK
env var in yatai deployment.
spec:
template:
spec:
containers:
env:
- name: YATAI_DONOT_TRACK
value: "true"