mage-ai / helm-charts Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
I tried deploying mage on AKS and followed the guide. Everything starts up happily and looking at the logs everything seems fine. However, when I try to access the external IP address I get a connection timeout. I'm guessing something is going on with the service definition, but I tried to alter it in various ways and I haven't been able to solve the problem.
I'm wondering if this is an isolated problem or if there is something wrong with the chart?
Hello. I saw that feature removed with #1. Isn't it possible to scale Mage? Why is that feature removed? Thanks!
The ingress is currently configured to forward to $svcPort
, which is the container port. This is wrong. It should be forwarding to port http
.
ingress.yaml
backend:
{{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
service:
name: {{ $fullName }}
port:
number: {{ $svcPort }}
{{- else }}
serviceName: {{ $fullName }}
servicePort: {{ $svcPort }}
{{- end }}
But the service maps the container port $svcPort
to http port.
service.yaml
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
Threfore the ingress uses an unreachable port on the service.
Correct would be probably something like this:
ingress.yaml
backend:
{{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
service:
name: {{ $fullName }}
port:
number: http
{{- else }}
serviceName: {{ $fullName }}
servicePort: http
{{- end }}
Hi,
how do you feel about introducing RWO volumes to the helm chart? I have seen mentions of RWO in workspaces, yet in the docs, EFS is required, so I'm not sure what is your opinion here.
Potentially, I can contribute the changes.
artifacthub.io/images: |
- name: mageai
image: mageai/mageai:0.9.16
This is the image version in chart.yaml can you guys update to latest @tommydangerous
Hello,
I have admin rights restricted to a single namespace in a kubernetes cluster, without the permission to use host-paths. When trying the installation as in https://mage-ai.github.io/helm-charts/, I get the errors below. Why is it required to deal with multiple namespaces, please ? Is there a version of the charts or a way to modify them in order to satisfy these constraints ?
Thanks
helm repo add mageai https://mage-ai.github.io/helm-charts
helm install my-mageai mageai/mageai
Error: INSTALLATION FAILED: 5 errors occurred:
* roles.rbac.authorization.k8s.io "mageai-job-manager" is forbidden: user "system:serviceaccount:projet-outil-ae:vscode-python-515304" (groups=["system:serviceaccounts" "system:serviceaccounts:projet-outil-ae" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:["batch"], Resources:["jobs/status"], Verbs:["create" "delete"]}
{APIGroups:["extensions"], Resources:["jobs"], Verbs:["create" "delete" "get"]}
{APIGroups:["extensions"], Resources:["jobs/status"], Verbs:["create" "delete" "get"]}
* roles.rbac.authorization.k8s.io "mageai-workspace-manager" is forbidden: user "system:serviceaccount:projet-outil-ae:vscode-python-515304" (groups=["system:serviceaccounts" "system:serviceaccounts:projet-outil-ae" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["ingresses"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:[""], Resources:["nodes"], Verbs:["watch" "create" "update" "patch" "delete"]}
{APIGroups:[""], Resources:["statefulsets"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:[""], Resources:["statefulsets/scale"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["apps"], Resources:["configmaps"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["apps"], Resources:["ingresses"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["apps"], Resources:["nodes"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["apps"], Resources:["persistentvolumeclaims"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["apps"], Resources:["services"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["configmaps"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["nodes"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["persistentvolumeclaims"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["services"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["statefulsets"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
{APIGroups:["networking.k8s.io"], Resources:["statefulsets/scale"], Verbs:["get" "watch" "create" "update" "patch" "delete"]}
* roles.rbac.authorization.k8s.io "mageai-job-manager" not found
* roles.rbac.authorization.k8s.io "mageai-workspace-manager" not found
* admission webhook "validate.kyverno.svc-fail" denied the request:
resource Deployment/projet-outil-ae/mageai was blocked due to the following policies
disallow-host-path:
autogen-host-path: 'validation error: HostPath volumes are forbidden. The field
spec.volumes[*].hostPath must be unset. rule autogen-host-path failed at path
/spec/template/spec/volumes/0/hostPath/'
Hello
I need to overwrite “busybox” image here:
context: (our infrastructure runs in an isolated env, meaning that any external image need to be pushed to our local registry)
When using the helm chart is possible to specify resouces based on the following section of values.yaml
standaloneScheduler: true
....
....
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
limits:
cpu: 100m
memory: 128Mi
requests:
cpu: 100m
memory: 128Mi
But, in this scenario both scheduler and webserver Pod (for standalone mode) use the same resouces.
I believe that would be helpful to set different resouces for scheduler and webserver, considering that websever pod requires less resouces to run than scheduler pods.
Chart version tested: 0.2.1
Thanks!
I think it would be helpful to allow dns configurations on helm chart like dnsConfig
and dnsPolicy
. For now, I need to set some custom configurations for dns and I'm not able to do this directly in helm chart. To solve this problem, I'm generating the yaml files using helm template
and deploying theses files. So, it would be helpful to set in helm chart, like this
dnsPolicy: "ClusterFirstWithHostNet"
dnsConfig:
nameservers:
- XXX.XXX.X.XXX
Due to the current helm template structure, the mageai-scheduler pod and the mageai-webserver pod are configured to refer to the same PVC.
---
title: mageai PVC architecture
---
flowchart LR
w("mageai-webserver")
s("mageai-scheduler")
pvc("mageai-pvc")
pv("mageai-pv")
w --"/home/src"--> pvc
s --"/home/src"--> pvc
pvc --> pv
# pod spec for mageai-webserver and mageai-scheduler
spec:
volumes:
- name: mage-fs
persistentVolumeClaim:
claimName: mageai-pvc
Need to refactor chart template for persistentVolumeClaim resource and volume spec
The indentation is broken in configmap.
values.yaml:
config:
FOO1: "F"
FOO2: "O"
FOO3: "O"
helm template mageai/mageai --values values.yaml --debug
[...]
Error: YAML parse error on mageai/templates/configmap.yaml: error converting YAML to JSON: yaml: line 12: did not find expected key
helm.go:84: [debug] error converting YAML to JSON: yaml: line 12: did not find expected key
YAML parse error on mageai/templates/configmap.yaml
[...]
Results in:
# Source: mageai/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: mageai-env
labels:
helm.sh/chart: mageai-0.1.5
app.kubernetes.io/name: mageai
app.kubernetes.io/instance: release-name
app.kubernetes.io/version: "0.9.44"
app.kubernetes.io/managed-by: Helm
data:
FOO1: F
FOO2: O
FOO3: O
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.