padok-team / burrito Goto Github PK

View Code? Open in Web Editor NEW

205.0 9.0 8.0 6.9 MB

🌯 Burrito is a TACoS 🌮

Home Page: https://padok-team.github.io/burrito

License: Apache License 2.0

Dockerfile 0.54% Makefile 2.12% Go 65.00% JavaScript 0.45% HTML 0.10% TypeScript 31.77% CSS 0.03%

cd ci cicd kubernetes kubernetes-operator operator terraform tacos

burrito's Introduction

burrito

Burrito is a TACoS (Terraform Automation Collaboration Software) Kubernetes Operator.

Why does this exists?

terraform is a tremendous tool to manage your infrastructure in IaC. However, it lacks built-in solutions for managing state drift.

Additionally, configuring a CI/CD pipeline for Terraform can be challenging and often varies depending on the selected tools

Finally, currently, there is no easy way to navigate your Terraform state to truly understand the modifications it undergoes when running terraform apply.

burrito aims to tackle those issues by:

Planning continuously your Terraform code and run applies if needed
Offering an out of the box PR/MR integration so you do not have to write CI/CD pipelines for Terraform ever again
Showing your state's modifications in a simple Web UI

Demo

Documentation

To learn more about burrito go to the complete documentation.

Community

Contibution, Discussion and Support

You can reach burrito's maintainers on Twitter:

Blogs and Presentations

Our burrito is a TACoS

License

Licensed under the Apache License, Version 2.0 (LICENSE)

burrito's People

Contributors

Stargazers

Watchers

Forkers

cterence imbaveteran tanandy hadrienpatte kalyann567 antverpp marcantoinegodde lucasmrqes

burrito's Issues

The controller triggers a plan when a push event is received

BUG: env.valueFrom in overrideRunnerSpec doesn't appear in pod spec

The merging logic we have make a list corev1.EnvVar without retrieving the valueFrom field.

Uniformize logging system and contexts

The controller supports redis authentication

The controller launches plan reconciliation every 20m

The controller supports Terragrunt code bases

We should implement a new runner type to handle terragrunt.

Also, we need to discuss wether terraform and terragrunt codebases will be handled through the same CRDs or with different CRDs

Improve status vizaulization with kubectl cli

Implement exponential back-off

Currently, if a terraform command fails. A new runner will be restarted indefinitely. We should implement an exponential back-off retry

Release flow and repository standardization

The controller can receive push events from Github

Improve manifests generation

The Makefile generates the CRDs in the config folder but does not update the manifests/all.yaml file

[Test] CLI flags overrides should be tested

As stated in PR #144, CLI flags are not unit tested.

They should take precedence over configuration files, but not over environment variables.

The controller integrates PR/MR workflow

We want the controller to be notified when a PR/MR is opened on a given layer. When it's the case, we want to generate a new TerraformLayer which will be temporary to run only the plans.

Ideally, the controller/runner should send the result of the plan as a comment in the MR/PR

The controller creates a new TerraformLayer on PullRequestOpen events

The storage capabilities supports multiple backends

Support for :

S3 Generic
S3 AWS (Auth)
GCS
Azure Blob Storage

Add unit testing

It would be nice to discuss in this thread which struct methods and functions need to be exposed outside their own packages

[Webhook] Testing environment

Setup a test environment around the Handle method of the Webhook object
Need to create a testenv first and load resources like terraformLayer controller
This will help test
- the additional paths feature
- pullrequest and push events

The Terraform Layer spec should use "ref" instead of "branch"

For a git repository, branch is a subset of ref which can include branch but also tags, commit, etc...

For argo app CRD, they use targetRevision instead.

Anyway, I think that naming this variable "ref" would still work for branch, but also enable some different uses cases such as testing on one commit or tag. Since it is a breaking change, it would be way easier to change it right away before there is any adoption.

However forcing the use of a branch can be an opinionated idea to push for a gitops/trunk workflow in all projects

Visitors can not understand easily the purpose of Burrito

Anyone who's coming on the repo can not understand the purpose of Burrito, a more complete readme could help :)

Make sure that burrito only watches resources in its specified namespaces

Make sure that burrito only watches the namespaces present in config

Use fixed versions in the dockerfile

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile

Dockerfile

docker.io/library/node 20.11.1@sha256:f3299f16246c71ab8b304d6745bb4059fa9283e8d025972e28436a9f9b36ed24

docker.io/library/golang 1.21.7@sha256:549dd88a1a53715f177b41ab5fee25f7a376a6bb5322ac7abe263480d9554021

docker.io/library/alpine 3.19.1@sha256:c5b1261d6d3e43071626931fc004f70149baeba2c8ec672bd4f27761f8e1ad6b

ui/Dockerfile

node 20

nginx stable-alpine

github-actions

.github/workflows/conventional-commits.yaml

actions/checkout v3

actions/setup-node v3

.github/workflows/docs.yaml

actions/checkout v4

actions/setup-python v4

actions/cache v3

.github/workflows/helm.yaml

actions/checkout v3

actions/checkout v3

.github/workflows/main.yaml

actions/checkout v3

actions/cache v3

actions/setup-go v4

codecov/codecov-action v3

actions/checkout v3

actions/setup-go v4

actions/checkout v3

actions/setup-go v4

golangci/golangci-lint-action v3

actions/checkout v3

docker/metadata-action v4

docker/setup-qemu-action v2

docker/setup-buildx-action v2

docker/login-action v2

docker/build-push-action v4

.github/workflows/pr.yaml

actions/checkout v3

actions/cache v3

actions/setup-go v4

codecov/codecov-action v3

actions/checkout v3

actions/setup-go v4

actions/checkout v3

actions/setup-go v4

golangci/golangci-lint-action v3

actions/checkout v3

docker/metadata-action v4

docker/setup-qemu-action v2

docker/setup-buildx-action v2

docker/login-action v2

docker/build-push-action v4

.github/workflows/release.yaml

actions/checkout v3

actions/setup-go v4

goreleaser/goreleaser-action v3

actions/checkout v3

stefanzweifel/git-auto-commit-action v4

actions/checkout v3

docker/metadata-action v4

docker/setup-qemu-action v2

docker/setup-buildx-action v2

docker/login-action v2

docker/build-push-action v4

gomod

go.mod

go 1.19

cloud.google.com/go/storage v1.40.0

github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.5.2

github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.3.2

github.com/aws/aws-sdk-go v1.51.25

github.com/bradleyfalzon/ghinstallation/v2 v2.8.0

github.com/hashicorp/go-multierror v1.1.1

github.com/hashicorp/terraform-json v0.17.1

github.com/onsi/ginkgo/v2 v2.13.2

github.com/onsi/gomega v1.29.0

github.com/sirupsen/logrus v1.9.3

github.com/stretchr/testify v1.9.0

google.golang.org/api v0.170.0

k8s.io/apimachinery v0.28.7

k8s.io/client-go v0.28.7

sigs.k8s.io/controller-runtime v0.15.3

github.com/bombsimon/logrusr/v4 v4.0.0

github.com/go-git/go-git/v5 v5.11.0

github.com/go-playground/webhooks v5.17.0+incompatible

github.com/go-redis/redis/v8 v8.11.5

github.com/google/go-cmp v0.6.0

github.com/google/go-github/v50 v50.2.0

github.com/hashicorp/go-version v1.6.0

github.com/hashicorp/hc-install v0.6.3

github.com/hashicorp/terraform-exec v0.19.0

github.com/labstack/echo/v4 v4.11.4

github.com/patrickmn/go-cache v2.1.0+incompatible

github.com/spf13/cobra v1.7.0

github.com/spf13/pflag v1.0.5

github.com/spf13/viper v1.16.0

github.com/xanzy/go-gitlab v0.93.2

golang.org/x/oauth2 v0.18.0

k8s.io/api v0.28.7

helm-values

deploy/charts/burrito/values.yaml

sealio/hermitcrab main

ghcr.io/padok-team/burrito

kustomize

manifests/base/kustomization.yaml

ghcr.io/padok-team/burrito main

npm

ui/package.json

@floating-ui/react ^0.26.9

@tanstack/react-query ^5.8.3

@tanstack/react-table ^8.10.7

axios ^1.5.1

react ^18.2.0

react-dom ^18.2.0

react-router-dom ^6.16.0

react-tooltip ^5.21.6

tailwind-merge ^2.0.0

@types/react ^18.2.15

@types/react-dom ^18.2.7

@typescript-eslint/eslint-plugin ^7.0.1

@typescript-eslint/parser ^7.0.1

@vitejs/plugin-react-swc ^3.3.2

autoprefixer ^10.4.16

eslint ^8.56.0

eslint-plugin-react-hooks ^4.6.0

eslint-plugin-react-refresh ^0.4.3

postcss ^8.4.31

tailwindcss ^3.3.3

typescript ^5.3.3

vite ^5.0.11

Check this box to trigger a request for Renovate to run again on this repository

The controller supports Gitlab equivalent events

The controller triggers a new plan reconciliation when a push event is received on an opened pull request

Automatically apply on specific changes

I thought about a specific use case: most of my Terraform codebase is somewhat idempotent. If you don't change anything in the code, after a successful apply all following plan/apply will find no changes to apply.

However I have some specific parts which might change, for example

a datasource for an AWS AMI, which default to the latest AMI available
a datasource containing a list of IP to block in the firewall. This datasource is managed by another team

For these specific changes, which can appear at any time, I would expect Burrito to notify me about this drift. However, I know that I can also blindy apply theses changes since it is common and mastered operations.

For other drifts however, it might be dangerous to apply blindly (for example reverting a manual hotfix in prod) and the planOnly mode of Burrito is more interesting.

I don't have an idea for an interface for this kind of configuration, but being able to have a selective "auto-apply" for know changes would we interesting

Create a validating webhook to validate resources before they are applied

We're seeing emerging usecases as Burrito grows in complexity :

Validating the remediationStrategy enum
Validating complex interaction between options

handle private modules

Currently, the runner will not be able to init a codebas eusing private terraform modules

Parametrize requeue time

Separate conditions to be able to distinguish apply needeed because of drift or because of push event

Currently, we use the same state to start an apply coming from a drift and a webhook event, we should seperate them

CI/CD pipelines: tests. build image

Runner image injected at runtime

Controller will re-launch plan and apply every time

The controller launches an apply reconciliation when the planned artifact and applied artifact are different

The runners use a custom docker image

Make ssh_know_hosts cm name configurable

The controller can connect to a private repository

Webhook improvements

Repository URL comparison

For GitHub events we compare the repository url to webUrl and the sshUrl received in the webhook.
We should be able to parse thos url, reducing the number of compraisons.
For GitLab, we do not comprae to any sshUrl at the moment

Multiple files triggering layer plan

Today our webhook only checks for change in the path of the layer
For Terraform codebases that have local modules we need to be able to specify those modules' paths
For Terragrunt it's especially important as there is multiple levels of inputs

Handle Terraform failure in runner

Currently a Terraform plan/apply failure in the runner does not result in an error.

It puts the runner.terraform.padok.cloud/plan-date or runner.terraform.padok.cloud/apply-date, resulting in the controller to believe that a plan was successfully generated for this layer (or apply successfully applied).

This code snippet should be adapted:

burrito/internal/runner/runner.go

Lines 83 to 96 in 6f45c0b

 case "plan": 

 sum, err = r.plan() 

 ann[annotations.LastPlanDate] = time.Now().Format(time.UnixDate) 

 if err == nil { 

 ann[annotations.LastPlanCommit] = commit 

 } 

 ann[annotations.LastPlanSum] = sum 

 case "apply": 

 sum, err = r.apply() 

 ann[annotations.LastApplyDate] = time.Now().Format(time.UnixDate) 

 ann[annotations.LastApplySum] = sum 

 if err == nil { 

 ann[annotations.LastApplyCommit] = commit 

 }

Also, and as stated in #27, runner should exit with a code greater than 0 when Terraform/Terragrunt fails inside the pod.

golang:alpine is not needed for the final stage of the Dockerfile
The --no-cache can be added to apk

Improve runner exiting

The runner should exit on error

The repository controller needs to check repository status

We need to check the repository status to handle the case where an event was not handled well

	case "plan":
	sum, err = r.plan()
	ann[annotations.LastPlanDate] = time.Now().Format(time.UnixDate)
	if err == nil {
	ann[annotations.LastPlanCommit] = commit
	}
	ann[annotations.LastPlanSum] = sum
	case "apply":
	sum, err = r.apply()
	ann[annotations.LastApplyDate] = time.Now().Format(time.UnixDate)
	ann[annotations.LastApplySum] = sum
	if err == nil {
	ann[annotations.LastApplyCommit] = commit
	}