azbuilder / terrakube-helm-chart Goto Github PK

View Code? Open in Web Editor NEW

30.0 30.0 19.0 402 KB

Helm chart to install Terrakube in any Kubernetes cluster

License: Apache License 2.0

terrakube-helm-chart's People

Contributors

Stargazers

Watchers

Forkers

scabarrus jz-wilson diliz jstewart612 robbert229 take-five marrau ilkerispir jpetrucciani kanzi xbuzz

terrakube-helm-chart's Issues

Deprecated annotations on k8s v1.27.4 using default helm install

Ingress classes seem to have deprecated annotations and paths

W1006 13:34:26.124574   43705 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
W1006 13:34:26.124600   43705 warnings.go:70] path /(.*) cannot be used with pathType Prefix
W1006 13:34:26.124605   43705 warnings.go:70] path /dex/(.*) cannot be used with pathType Prefix

K8S Version

serverVersion:
  buildDate: "2023-07-19T12:14:49Z"
  compiler: gc
  gitCommit: fa3d7990104d7c1f16943a67f11b154b71f6a132
  gitTreeState: clean
  gitVersion: v1.27.4
  goVersion: go1.20.6
  major: "1"
  minor: "27"
  platform: linux/amd64

Setup

minikube start \
--driver="docker" \
--memory="4G" \
--cpus="4" --addons=ingress --addons=storage-provisioner

helm install terrakube terrakube-repo/terrakube -n terrakube --create-namespace

Helm result

W1006 13:34:26.124574   43705 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
W1006 13:34:26.124600   43705 warnings.go:70] path /(.*) cannot be used with pathType Prefix
W1006 13:34:26.124605   43705 warnings.go:70] path /dex/(.*) cannot be used with pathType Prefix
W1006 13:34:26.178316   43705 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
W1006 13:34:26.178330   43705 warnings.go:70] path /(.*) cannot be used with pathType Prefix
W1006 13:34:26.178343   43705 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
W1006 13:34:26.178355   43705 warnings.go:70] path /(.*) cannot be used with pathType Prefix
NAME: terrakube
LAST DEPLOYED: Fri Oct  6 13:34:23 2023
NAMESPACE: terrakube
STATUS: deployed
REVISION: 1
TEST SUITE: None

Deployment example if use single domain for all components (UI/API/Dex/Registry) ingresses + storage backend authentication using roles + jdbc connection url with ssl

Trying to setup all components (UI/API/Dex/Registry) ingresses using same domain (terrakube.mycompany.com) and prefix-based routing.
UI ingress path: terrakube.mycompany.com/ui
API ingress path: terrakube.mycompany.com/api
Dex ingress path: terrakube.mycompany.com/dex
Registry ingress path: terrakube.mycompany.com/reg

However, the ingresses are not working. Does terrakube support prefix-based routing?

Created postgressql cluster using Zalando postgres operator (https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml) Terrakube postgressql connection fails with no encryption if connected to external postgres database
Caused by: liquibase.exception.DatabaseException: org.postgresql.util.PSQLException: FATAL: pg_hba.conf rejects connection for host "10.xxx.xxx.xx", user "terrakube", database "terrakube", no encryption
How to pass or override jdbc connection url?
Is it possible to configure IAM role instead of access key and secret key for AWS S3 storage if runs in EKS kubernetes cluster?

Deployment example values.yaml for Google Kubernetes Engine

Create example values.yaml to deploy Terrakube in GKE

Add support for GCP Storage backend

Terrakube 2.5.0 added support to use gcp storage as backend for state, output and modules

Reference:
AzBuilder/terrakube#216

Add Version Parameter

Add a parameter to manage version, because now all the components are using the same version number

Use an alterntavie for Snippet directives

Find an alternative for snippet directives when installing using minikube to forward the auth token

Related to AzBuilder/terrakube#618

Configure Liveness, Readiness and Startup Probes

These PR added the following endpoint to help monitor the api, registry and executor:

/actuator/health
/actuator/health/liveness
/actuator/health/readiness

The following templates need to be updated:

Kubernetes reference:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Spring boot references:
https://docs.spring.io/spring-boot/docs/2.6.8/reference/html/actuator.html#actuator.enabling

Incompatible with Google Certificates on GKE

The use of the ingress.useTls value to control both the Ingress TLS spec and the scheme of URLs in the app config make this chart not work with Google Certificates on the GCE-native Ingress. This is because annotations are used to attach Google managed or self-managed certificates to an Ingress resource.

If you set ingress.useTls to true the Ingresses will fail to create GCE Load Balancers because the TLS secret does not exist, and if you set it to false the Oauth redirect URI will use HTTP instead of HTTPS. Allowing independent control of the Ingress TLS spec and the relevant portions of the app config should resolve.

gh-pages helm repo

Hello there!

Can you add gh-pages to your repo so we can use it as a helm repository? :)

As an example on how to set it up:
https://helm.sh/docs/howto/chart_releaser_action/

An example repository where it's already done:
https://github.com/prometheus-community/helm-charts

Thanks for your hard work ;)

Login not working

Hello,

today i deployed Terrakube on an AKS Cluster by pretty much following the example AzureAuthentication-Example1.

I´ve created an Azure AD Group called terrakube-admin-access, created an Azure AD Application, and a Storage-Account.
In my values.yaml I´ve added the Storage Account Name, aswell ass the Ressource Group and the Access Key of it
Within the connectors section of dex I´ve added the Client ID, the Tenant ID and the Client-Secret of my Azure AD Application,

However, after deploying the Chart, clicking on the Login button from the UI does nothing.
Unfortunately i do not see any log message in any of the components.
I was hoping that someone could point out what is missing in my setup.

Here is my values.yaml

## Terrakube Security
security:
  useOpenLDAP: false
  adminGroup: "terrakube-admin-access"
  patSecret: "<<REDACTED>>"
  internalSecret: "<<REDACTED>>"
  dexClientId: "microsoft"
  dexClientScope: "email openid profile offline_access groups"
  dexIssuerUri: "https://terrakube-api.my-domain.internal/dex"

## Terraform Storage
storage:
  defaultStorage: false
  azure:
    storageAccountName: "<<REDACTED>>"
    storageAccountResourceGroup: "<<REDACTED>>"
    storageAccountAccessKey: "<<REDACTED>>"

## Dex
dex:
  enabled: true
  version: "v2.32.0"
  replicaCount: "1"
  serviceType: "ClusterIP"
  useOpenLDAP: false
  config:
    issuer: https://terrakube-api.my-domain.internal/dex
    storage:
      type: memory
    oauth2:
      responseTypes: ["code", "token", "id_token"] 
      skipApprovalScreen: true
    web:
      allowedOrigins: ['*']
      logger:
    staticClients:
    - id: microsoft
      redirectURIs:
      - 'https://terrakube.my-domain.internal'
      - 'http://localhost:3000'
      - 'http://localhost:10001/login'
      - 'http://localhost:10000/login'
      - '/device/callback'
      name: 'microsoft'
      public: true
    connectors:
    - type: microsoft
      id: microsoft
      name: microsoft
      config:
        clientID: "<<REDACTED>>"
        clientSecret: "<<REDACTED>>"
        redirectURI: "https://terrakube-api.my-domain.internal/dex/callback"
        tenant: "<<REDACTED>>"

## API properties
api:
  enabled: true
  defaultDatabase: false
  version: "2.10.0"
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    databaseType: "H2"

## Executor properties
executor:
  enabled: true
  version: "2.10.0"  
  replicaCount: "1"
  serviceType: "ClusterIP"
  properties:
    toolsRepository: "https://github.com/AzBuilder/terrakube-extensions"
    toolsBranch: "main"

## Registry properties
registry:
  enabled: true
  version: "2.10.0"
  replicaCount: "1"
  serviceType: "ClusterIP"

## UI Properties
ui:
  enabled: true
  version: "2.10.0"
  replicaCount: "1"
  serviceType: "ClusterIP"

## Ingress properties
ingress:
  useTls: true
  ui:
    enabled: true
    domain: "terrakube.my-domain.internal"
    path: "/(.*)"
    pathType: "Prefix" 
    annotations:
      nginx.ingress.kubernetes.io/use-regex: "true"
      kubernetes.io/ingress.class: "nginx"
      nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
      cert-manager.io/cluster-issuer: "my-domain-internal"
      external-dns/zone: "my-domain.internal"
  api:
    enabled: true
    domain: "terrakube-api.my-domain.internal"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: "nginx"
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
      cert-manager.io/cluster-issuer: "my-domain-internal"
      external-dns/zone: "my-domain.internal"
  registry:
    enabled: true
    domain: "terrakube-reg.my-domain.internal"
    path: "/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: "nginx"
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
      cert-manager.io/cluster-issuer: "my-domain-internal"
      external-dns/zone: "my-domain.internal"
  dex:
    enabled: true
    path: "/dex/(.*)"
    pathType: "Prefix"
    annotations:
      kubernetes.io/ingress.class: "nginx"
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: "proxy_set_header Authorization $http_authorization;"
      nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
      cert-manager.io/cluster-issuer: "my-domain-internal"
      external-dns/zone: "my-domain.internal"

Deployment example values.yaml for Azure Kuberentes Service

Create an example values.yaml to deploy the Terrakube using AKS using Application Gateway Ingress Controller.

Reference

Automatically Roll Deployments

Implement this feature in the helm chart

https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments

Add support for Dex

The following PR AzBuilder/terrakube#231 added support to handle the authentication using Dex

Taints and Tolerations

Add support for toleration

Reference: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/

terrakube with keycloak authentication

Hello,

We try to use terrakube with dex configuration to use openid connect authentication via keycloak.

We use for each components an internal pki to provide certificate signed per our CA.

We import the certificate of our IdP in dex in folder /etc/ssl/certs and also in each other pod to see if it solve my issue.

We are able to authenticate with admin role, but when we try to create an organization, We have these errors:

via the browser:
GET https://terrakube-api.xxxxxx/api/v1/organization 500
via pod output:
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

It seems the certificate need to be imported in keystore, but as I'm not fluent in Java, I would like to know if you have some guidance to help me.

Best regards,

terrakube default values should work out of the box

Hello there!

The helm chart should be installable as it is in a simple kubernetes environment, like minikube, kind, or whatever clusters.

It would provide a better "accessibility" to this tool, and facilitate adoption, I currently had to check the templates and values.schema.json to make it work, should not really be the case for "default" kubernetes context (like quick homelabs stuff)

Can you provide a "sandbox" version of the values that would work just by using helm install as this?

EDIT:

By doing:

helm repo add terrakube-community https://AzBuilder.github.io/terrakube-helm-chart
helm repo update
helm install terrakube-community/terrakube -n terrakube --generate-name

I got the following output:

Error: INSTALLATION FAILED: values don't meet the specifications of the schema(s) in the following chart(s):
terrakube:
- ui.serviceType: ui.serviceType must be one of the following: "ClusterIP", "NodePort", "LoadBalancer", "ExternalName"
- ingress.ui.pathType: ingress.ui.pathType must be one of the following: "ImplementationSpecific", "Exact", "Prefix"
- ingress.api.pathType: ingress.api.pathType must be one of the following: "ImplementationSpecific", "Exact", "Prefix"
- ingress.registry.pathType: ingress.registry.pathType must be one of the following: "ImplementationSpecific", "Exact", "Prefix"
- registry.serviceType: registry.serviceType must be one of the following: "ClusterIP", "NodePort", "LoadBalancer", "ExternalName"
- security: patSecret is required
- security: internalSecret is required
- api.properties.databaseType: api.properties.databaseType must be one of the following: "H2", "SQL_AZURE", "POSTGRESQL", "MYSQL"
- api.serviceType: api.serviceType must be one of the following: "ClusterIP", "NodePort", "LoadBalancer", "ExternalName"
- executor.serviceType: executor.serviceType must be one of the following: "ClusterIP", "NodePort", "LoadBalancer", "ExternalName"