mondoohq / mondoo-operator Goto Github PK

View Code? Open in Web Editor NEW

33.0 8.0 12.0 2.75 MB

☸️ Mondoo Client Kubernetes Operator

Home Page: https://mondoo.com

License: Other

Dockerfile 0.19% Makefile 2.60% Go 96.60% Smarty 0.24% Shell 0.29% HCL 0.07%

kubernetes kubernetes-operator security security-audit assessment operator

mondoo-operator's People

Contributors

Stargazers

Watchers

Forkers

cuddletech misterpantz tas50 jsoref presentjay rafaelbattesti ippx stnert mariuskimmina luis-sousa-pinto reachaxis

mondoo-operator's Issues

Test and document deployment in GKE cluster

To make it easier to use Mondoo with GKE, we want to add instruction how to deploy Mondoo into GKE.

test setup of the operator in GKE
document step how to best install it into GKE and add this to our /docs

Scan container images of running container

To include the scanning of the container images from running container, we need to adapt the inventory to include the discovery flag container-images

apiVersion: v1
kind: Inventory
metadata:
  name: mondoo-k8s-inventory
  labels:
    environment: production
spec:
  assets:
    - id: minikube
      connections:
        - backend: k8s
          discover:
            targets:
              - container-images

configurable k8s secret in MondooConfig

We externalised the mondoo credential into a kube secret via #38. At this point we use a hard-coded name for the secret. Going forward we want to be able to configure the used secret via the MondooConfig

Mondoo Config should be watched in operator namespace only

In preparation for #31 we need to make sure the operator only looks for CRD instances of Mondoo Config in the same namespace.

This may already work this way.

E2E testing OLM deployment

What is is not working as you expected it?
operator-sdk run bundle tries to pull the referenced image b4 checking local repo.
For PRs this will always fail.

Where on the platform does it happen?
mondoo-operator

How do we replicate the issue?
Add E2E testing with operator-sdk run bundle and you'll get a 404

Expected behavior (i.e. solution)
We should be able to use a local bundle image for e2e testing.

Other Comments
Might be a issue on operator-sdk's github

Helm Chart to deploy operator

Improve OLM Bundle Release

To ensure a continuous release, tagged releases should:

build and release the olm bundle
run e2e test of bundle after the image has been published (my understanding is that it is difficult to test before we have it published, then lets at least do this quickly after)
remove generated /bundle content from git and only publish the content into the release (since this is always autogenerated we do not need to commit it)

Move mondoo credentials into secret

To avoid leaking the credentials, we should move them out of Mondoo Config and deploy them separately. Users could upload the secret before deployment with:

kubectl create secret generic mondoo-credentials --namespace kube-system --from-file=mondoo.yml=mondoo.yml

configmap --> secret

Currently we store the mondoo config as a configmap, this needs to be a secret.

Simplify build pipeline by building the docker image only once

Right now, we building the docker image twice for normal build and for e2e test and it takes way too long.

We should build the container image once and push it as mondoolabs/mondoo-operator:gitsha

The e2e test is using the image with the sha. On tagged release we also push them as:

mondoolabs/mondoo-operator:tag
mondoolabs/mondoo-operator:latest

Another improvement option is to build for x64 first, run the tests and then build the missing architectures only for released tags

Add installation instructions for kubectl

We want to make it easy for users to setup the operator. Therefore we need to add operator installation description. It should include:

instructions to setup the operator with kubectl
required permissions
uninstall instructions

# create secret
kubectl create secret generic mondoo-credentials --namespace kube-system --from-file=mondoo.yml=mondoo.yml

# deploy operator with default configuration
kubectl apply -f operator.yml

Since we extracted the mondoo service account into its own secret via #25, we can deploy the operator with default settings with one call.

We should update the make file to generate a single Kubernetes manifest. We probably need a different make task to generate a joint yml file that includes crd and config https://github.com/mondoolabs/mondoo-operator/blob/main/Makefile#L107-L118

Use of return: requeue true in the reconciler method

Is your feature request related to a problem? Please describe.
Operator-sdk scaffolded a structure using return requeue, we've removed most of these calls to get the operator working for us. We feel this is not best practice.

Describe the solution you'd like
We would like the strucutre to be closer to what operator-sdk originally scaffolded for the memcach example.

Describe alternatives you've considered
na

Additional context
na

Publish as operator bundle

We need to integrate the operator with OLM: https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/

Improve Helm Release process

In #84 we worked on releasing helm into GitHub pages. Since the release is then covered, we can optimise the repository:

lets remove all auto-generated content from the git repository
add the generated chart to the GitHub action release task
publish the bundle (should be already covered by #84)

Test and document deployment into AKS

To make it easier to use Mondoo with Azure AKS, we want to add instruction how to deploy Mondoo into AKS.

test setup of the operator in AKS
document step how to best install it into AKS and add this to our /docs

Github action for test

We want to ensure all code is properly tested. Therefore we want to start extending continuous testing via make test in GitHub actions.

Reference:

prometheus has embedded the test in GitHub actions: https://github.com/prometheus-operator/prometheus-operator/blob/main/.github/workflows/unit.yaml

Installation on M1 Macs

Bug Report
Unable to Build Operator on Arm64 architecture MBP 2021
What did you do?
Cloned mondoo-operator repo and ran make test
What did you expect to see?
PASS
What did you see instead? Under which circumstances?

go: creating new go.mod: module tmp
Downloading sigs.k8s.io/controller-tools/cmd/[email protected]
go get: installing executables with 'go get' in module mode is deprecated.
	To adjust and download dependencies of the current module, use 'go get -d'.
	To install using requirements of the current module, use 'go install'.
	To install ignoring the current module, use 'go install' with a version,
	like 'go install example.com/cmd@latest'.
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.
go get: added github.com/fatih/color v1.12.0
go get: added github.com/go-logr/logr v0.4.0
go get: added github.com/gobuffalo/flect v0.2.3
go get: added github.com/gogo/protobuf v1.3.2
go get: added github.com/google/go-cmp v0.5.6
go get: added github.com/google/gofuzz v1.1.0
go get: added github.com/inconshreveable/mousetrap v1.0.0
go get: added github.com/json-iterator/go v1.1.11
go get: added github.com/mattn/go-colorable v0.1.8
go get: added github.com/mattn/go-isatty v0.0.12
go get: added github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go get: added github.com/modern-go/reflect2 v1.0.1
go get: added github.com/spf13/cobra v1.2.1
go get: added github.com/spf13/pflag v1.0.5
go get: added golang.org/x/mod v0.4.2
go get: added golang.org/x/net v0.0.0-20210520170846-37e1c6afe023
go get: added golang.org/x/sys v0.0.0-20210616094352-59db8d763f22
go get: added golang.org/x/text v0.3.6
go get: added golang.org/x/tools v0.1.5
go get: added golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go get: added gopkg.in/inf.v0 v0.9.1
go get: added gopkg.in/yaml.v2 v2.4.0
go get: added gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
go get: added k8s.io/api v0.22.2
go get: added k8s.io/apiextensions-apiserver v0.22.2
go get: added k8s.io/apimachinery v0.22.2
go get: added k8s.io/klog/v2 v2.9.0
go get: added k8s.io/utils v0.0.0-20210819203725-bdf08cb9a70a
go get: added sigs.k8s.io/controller-tools v0.7.0
go get: added sigs.k8s.io/structured-merge-diff/v4 v4.1.2
go get: added sigs.k8s.io/yaml v1.2.0
/Users/harsha/mondoo-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/harsha/mondoo-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go: creating new go.mod: module tmp
Downloading sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
go get: installing executables with 'go get' in module mode is deprecated.
	To adjust and download dependencies of the current module, use 'go get -d'.
	To install using requirements of the current module, use 'go install'.
	To install ignoring the current module, use 'go install' with a version,
	like 'go install example.com/cmd@latest'.
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.
go get: added github.com/go-logr/logr v1.2.0
go get: added github.com/go-logr/zapr v1.2.0
go get: added github.com/spf13/afero v1.6.0
go get: added github.com/spf13/pflag v1.0.5
go get: added go.uber.org/atomic v1.7.0
go get: added go.uber.org/multierr v1.6.0
go get: added go.uber.org/zap v1.19.1
go get: added golang.org/x/text v0.3.6
go get: added sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20220112150637-b8db76e66383
unable to find a version that was supported for platform darwin/arm64
KUBEBUILDER_ASSETS="" go test ./... -coverprofile cover.out
?   	github.com/mondoolabs/mondoo-operator	[no test files]
?   	github.com/mondoolabs/mondoo-operator/api/v1alpha1	[no test files]
Running Suite: Controller Suite
===============================
Random Seed: 1642004521
Will run 0 of 0 specs

STEP: bootstrapping test environment
2022-01-12T17:22:01.348+0100	DEBUG	controller-runtime.test-env	starting control plane
2022-01-12T17:22:01.353+0100	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 0, "error": "exec: \"etcd\": executable file not found in $PATH"}
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238
github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221
github.com/mondoolabs/mondoo-operator/controllers.TestAPIs
	/Users/harsha/mondoo-operator/controllers/suite_test.go:47
testing.tRunner
	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259
2022-01-12T17:22:01.354+0100	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 1, "error": "exec: \"etcd\": executable file not found in $PATH"}
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238
github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221
github.com/mondoolabs/mondoo-operator/controllers.TestAPIs
	/Users/harsha/mondoo-operator/controllers/suite_test.go:47
testing.tRunner
	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259
2022-01-12T17:22:01.354+0100	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 2, "error": "exec: \"etcd\": executable file not found in $PATH"}
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238
github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221
github.com/mondoolabs/mondoo-operator/controllers.TestAPIs
	/Users/harsha/mondoo-operator/controllers/suite_test.go:47
testing.tRunner
	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259
2022-01-12T17:22:01.355+0100	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 3, "error": "exec: \"etcd\": executable file not found in $PATH"}
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238
github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221
github.com/mondoolabs/mondoo-operator/controllers.TestAPIs
	/Users/harsha/mondoo-operator/controllers/suite_test.go:47
testing.tRunner
	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259
2022-01-12T17:22:01.355+0100	ERROR	controller-runtime.test-env	unable to start the controlplane	{"tries": 4, "error": "exec: \"etcd\": executable file not found in $PATH"}
github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113
github.com/onsi/ginkgo/internal/leafnodes.(*runner).run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64
github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runBeforeSuite
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:123
github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:63
github.com/onsi/ginkgo/internal/suite.(*Suite).Run
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79
github.com/onsi/ginkgo.runSpecsWithCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238
github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters
	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221
github.com/mondoolabs/mondoo-operator/controllers.TestAPIs
	/Users/harsha/mondoo-operator/controllers/suite_test.go:47
testing.tRunner
	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259
Failure [0.007 seconds]
[BeforeSuite] BeforeSuite
/Users/harsha/mondoo-operator/controllers/suite_test.go:52

  Unexpected error:
      <*fmt.wrapError | 0x140004a4860>: {
          msg: "unable to start control plane itself: failed to start the controlplane. retried 5 times: exec: \"etcd\": executable file not found in $PATH",
          err: <*fmt.wrapError | 0x140004a4820>{
              msg: "failed to start the controlplane. retried 5 times: exec: \"etcd\": executable file not found in $PATH",
              err: <*exec.Error | 0x140004a47c0>{
                  Name: "etcd",
                  Err: <*errors.errorString | 0x14000366f10>{
                      s: "executable file not found in $PATH",
                  },
              },
          },
      }
      unable to start control plane itself: failed to start the controlplane. retried 5 times: exec: "etcd": executable file not found in $PATH
  occurred

  /Users/harsha/mondoo-operator/controllers/suite_test.go:62
------------------------------
STEP: tearing down the test environment
Panic [0.000 seconds]
[AfterSuite] AfterSuite
/Users/harsha/mondoo-operator/controllers/suite_test.go:76

  Test Panicked
  runtime error: invalid memory address or nil pointer dereference
  /opt/homebrew/Cellar/go/1.17.5/libexec/src/runtime/panic.go:221

  Full Stack Trace
  sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane.(*APIServer).Stop(0x140001be0e0)
  	/Users/harsha/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/testing/controlplane/apiserver.go:417 +0x24
  sigs.k8s.io/controller-runtime/pkg/internal/testing/controlplane.(*ControlPlane).Stop(0x140001e8000)
  	/Users/harsha/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/testing/controlplane/plane.go:87 +0x40
  sigs.k8s.io/controller-runtime/pkg/envtest.(*Environment).Stop(0x140001e8000)
  	/Users/harsha/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:194 +0xfc
  github.com/mondoolabs/mondoo-operator/controllers.glob..func2()
  	/Users/harsha/mondoo-operator/controllers/suite_test.go:78 +0x50
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0x140001134a0)
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:113 +0x9c
  github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0x140001134a0)
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:64 +0xe8
  github.com/onsi/ginkgo/internal/leafnodes.(*simpleSuiteNode).Run(0x140001806e0, 0x1, 0x1, {0x0, 0x0})
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/suite_nodes.go:25 +0x64
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runAfterSuite(0x14000166dc0)
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:138 +0x90
  github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0x14000166dc0)
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/specrunner/spec_runner.go:71 +0xc8
  github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0x14000393490, {0x10342b978, 0x14000212b60}, {0x101cb0ece, 0x10}, {0x1400007f9a0, 0x2, 0x2}, {0x10213cc78, 0x14000467780}, ...)
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/internal/suite/suite.go:79 +0x3e0
  github.com/onsi/ginkgo.runSpecsWithCustomReporters({0x1021002c0, 0x14000212b60}, {0x101cb0ece, 0x10}, {0x1400007f980, 0x2, 0x2})
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:238 +0x13c
  github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters({0x1021002c0, 0x14000212b60}, {0x101cb0ece, 0x10}, {0x140004f8758, 0x1, 0x1})
  	/Users/harsha/go/pkg/mod/github.com/onsi/[email protected]/ginkgo_dsl.go:221 +0x180
  github.com/mondoolabs/mondoo-operator/controllers.TestAPIs(0x14000212b60)
  	/Users/harsha/mondoo-operator/controllers/suite_test.go:47 +0xe4
  testing.tRunner(0x14000212b60, 0x1020f4e78)
  	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1259 +0x104
  created by testing.(*T).Run
  	/opt/homebrew/Cellar/go/1.17.5/libexec/src/testing/testing.go:1306 +0x328
------------------------------


Ran 0 of 0 Specs in 0.007 seconds
FAIL! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
You're using deprecated Ginkgo functionality:
=============================================
Ginkgo 2.0 is under active development and will introduce (a small number of) breaking changes.
To learn more, view the migration guide at https://github.com/onsi/ginkgo/blob/v2/docs/MIGRATING_TO_V2.md
To comment, chime in at https://github.com/onsi/ginkgo/issues/711

  You are using a custom reporter.  Support for custom reporters will likely be removed in V2.  Most users were using them to generate junit or teamcity reports and this functionality will be merged into the core reporter.  In addition, Ginkgo 2.0 will support emitting a JSON-formatted report that users can then manipulate to generate custom reports.

  If this change will be impactful to you please leave a comment on https://github.com/onsi/ginkgo/issues/711
  Learn more at: https://github.com/onsi/ginkgo/blob/v2/docs/MIGRATING_TO_V2.md#removed-custom-reporters

To silence deprecations that can be silenced set the following environment variable:
  ACK_GINKGO_DEPRECATIONS=1.16.4

--- FAIL: TestAPIs (0.01s)
FAIL
coverage: 0.0% of statements
FAIL	github.com/mondoolabs/mondoo-operator/controllers	0.287s
FAIL
make: *** [test] Error 1

Environment

Kubernetes cluster type:
minikube

$ operator-sdk version
operator-sdk version: "v1.15.0", commit: "f6326e832a8a5e5453d0ad25e86714a0de2c0fc8", kubernetes version: "v1.21", go version: "go1.17.5", GOOS: "darwin", GOARCH: "arm64"

This bug is possibly related to an underlying open issue(operator-framework/operator-sdk#5090) with kubebuilder. Operator SDK depends on kubebuilder

Define required metrics to expose it for prometheus

research best patterns from known operators
define the metrics that are applicable for mondoo
add logic to collect the metrics to the operator

Generate Helm manifest files using Kustomize

What is is requested?
Generate Helm manifests from kustomize files to reduce human error while creating helm chart

Other Comments

setup base operator-sdk

[email protected]

operator-sdk init --domain mondoo.io --repo github.com/mondoolabs/mondoo-operator

Switch disable config to enable config

switch config from:

  kubeapi:
    disable: false
  kubenodes:
    disable: false

to:

  kubeapi:
    enable: true 
  kubenodes:
    enable: true

Release Helm Chart

Is your feature request related to a problem? Please describe.
Currently helm chart can only be installed by cloning the operator repository. Fix this by hosting helm chart at an appropriate place.

Describe the solution you'd like
Current method to install helm chart

 helm install mondoo-operator ./chart --namespace mondoo-operator-system

Expected way to install

helm repo add mondoo [url]
helm repo update
helm install [Release_Name] mondoo/mondoo-operator
     
**Additional context**
https://helm.sh/docs/howto/chart_releaser_action/#helm
https://helm.sh/docs/topics/chart_repository/#github-pages-example

Reference github package repository in Makefile

Is your feature request related to a problem? Please describe.
We need the makefile to reference the remote docker repository as base like argo is doing it here to avoid conflicts in the CI:

https://github.com/argoproj-labs/argocd-operator/blob/master/Makefile

Describe the solution you'd like
Configure our makefile to match argo's layout

Describe alternatives you've considered
na

Additional context
na

🐛 container reference in mondoo-operator-manifests.yaml incorrect

Describe the bug

It seems like the container image reference we use in https://github.com/mondoohq/mondoo-operator/releases/download/v0.0.8/mondoo-operator-manifests.yaml is not correct.

It refers to a version that does not exist: ghcr.io/mondoohq/mondoo-operator:sha256-3c4c89d8f0aed1c7a567996c256829ac6be0bdb1.sig

Also the .sig is wrong since this is referring to the signature. Therefore the kubectl installation is not working right now until the image is change to ghcr.io/mondoohq/mondoo-operator:latest

To Reproduce

Follow updated instructions for kubectl in https://github.com/mondoohq/mondoo-operator/pull/116/files#diff-0aa9a4cedf3c435cfb032f9064ae66dce0406584c093b598e5222b297fe563f5

Expected behavior

kubectl installation just works

Screenshots

n/a

Desktop (please complete the following information):

minikube

Additional context
Add any other context about the problem here.

Reconcile loop should also handle existing configuration

as mentioned in #23

Currently the reconcile loop handles only new objects. We nee to make sure the existing configuration is either teared down or transformed to the expected state.

add contribution guidelines

What is is not working as you expected it?

Add general contribution guidelines

Where on the platform does it happen?

n/a

How do we replicate the issue?

n/a

Expected behavior (i.e. solution)

n/a

Other Comments

n/a

Add health checks to Mondoo Client pods

Is your feature request related to a problem? Please describe.
Current policy only checks for existence of deployment and daemonset. Needs to be extended to check if the containers have started up correctly.

Describe the solution you'd like
Use Mondoo to scan container runtime logs files for correct startup messages.

Describe alternatives you've considered
Just parse files using awk and compare . It is cumbersome . Better to include as part of policy.

Additional context
Mondoo currently does not allow parsing journalctl files

Github action for e2e test

We want to make sure that our operator works with each change. Therefore we want to run a full e2e test in Github Actions. The goal is to build and deploy the operator in the pipeline.

Reference:

Prometheus has a interesting setup with kind: https://github.com/prometheus-operator/prometheus-operator/blob/main/.github/workflows/e2e.yaml

Test and document deployment into EKS

To make it easier to use Mondoo with Amazon EKS, we want to add instruction how to deploy Mondoo into EKS.

test setup of the operator in EKS
document step how to best install it into EKS and add this to our /docs

Add documentation for Roles and Permissions for Application Scanner

Is your feature request related to a problem? Please describe.
Add documentation for Permissions required to scan API server.

Document how to setup multiple MondooAuditConfig's in different namespaces

We want to be able to report individual namespaces to individual spaces:

ability to use multiple mondoo credentials
configure Mondoo config in a specific namespace and allow users to set a credential
default should stay cluster-wide scan (all namespaces)
nodes scan only makes sense for cluster-wide deployment, we would not allow nodes scan restricted to a namespace

In order to support that:

we want to ensure the namespace is not hardcoded
secret need to be stored into the same namespace as the MondooConfig
our kubectl setup always setup a default namespace for the operator

Determine requirements to publish to GKE marketplace

Google has a public marketplace https://cloud.google.com/marketplace/docs/kubernetes-apps and Mondoo should be easy to be activated via that. We need to determine the requirements to bring the operator into the marketplace.

Operator-sdk Get calls are not working as expected

What is is not working as you expected it?
The operator-sdk get call syntax is not returning nil on existing resources, instead it returns empty string error --> in source code this is stated as the equivalent of 500 server errror.

Where on the platform does it happen?
Operator controller.

How do we replicate the issue?
Controller resource updates are not executed when we use the operator-sdk syntax.

Expected behavior (i.e. solution)
The Get statement should return nil on existing resources not an error.

Other Comments
We fixed this by changing

err = r.Get(ctx, types.NamespacedName{Name: inventoryDaemonSet, Namespace: mondoo.Namespace}, foundConfigMap)

-->

err = r.Get(ctx, client.ObjectKeyFromObject(&corev1.ConfigMap{
			ObjectMeta: metav1.ObjectMeta{
				Name:      inventoryDaemonSet,
				Namespace: mondoo.Namespace,
			},
		}), foundConfigMap)

Rename MondooClient to MondooAuditConfig

MondooConfig is very generic, changing the name to MondooAuditConfig makes it intention more clear and also prepares for #31

Improve config names

I was thinking about the current naming

MondooConfig: very generic
kubeNodes not clear what this refers to
kubeAPI as well

What if we structure this under MondooAuditConfig. Then it is clear that everything is related to Audit. Then we can use resources and nodes

apiVersion: k8s.mondoo.com/v1alpha1
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator-system
data:
  mondooSecretRef: "name"
  resources:
    enable: true
  nodes:
    enable: true

Simplify audit config

With #54 we renamed the configuration to MondooAuditConfig which is great but the properties names can be optimised. A current sample config looks like:

apiVersion: k8s.mondoo.com/v1alpha1
kind: MondooAuditConfig
metadata:
  name: mondoo-client
  namespace: mondoo-operator-system
spec:
  workloads:
    enable: true
    workloadserviceaccount: mondoo-operator-workload
    replicas: 1
  nodes:
    enable: true
  mondoosecretref: mondoo-client

The following entries are confusing:

k8s uses camelCase for yaml attributes as default, we should adapt
workloadserviceaccount should be renamed to serviceAccountName since it is already part of the workload property
replicas should be removed for now since the Mondoo does not support replicas yet. Right now the replica is always one. We are going to re-introduce that at a later point when we have a proper queuing mechanism.
mondoosecretref should be renamed to mondooSecretRef

setup 2 node gke cluster @endocde for testing

change go mod package to

currently go.mod uses an package name that we do not use. We need to switch from:

github.com/mondoolabs/mondoo-operator

go.mondoo.com/mondoo-operator

Publish Operator on Docker Hub

Docker Build Pipeline is failing

The docker build pipeline is failing

Add readme instructions to get started

For new users it should have clear instructions how to get started.

The section should have 3 ways documented to install the operator:

Manual with kubectl
OLM
Helm Chart

Publish generated kubectl yaml file for tagged releases

adapt github action to publish the generated yaml file

extract ClusterRole, ClusterRoleBinding, ServiceAccount into standard setup instead of being part of the Reconcile loop

As discovered in #23 (comment) we currently dynamically update the security configuration. This should not be done. Therefore we want to make the ClusterRole, ClusterRoleBinding and the ServiceAccount part of the default configuration.

Multi-Arch Container Build to support ARM

We want to release the operator for the following architectures:

arm64
armv7

This allows us to test and deploy the operator to 32 bit / 64 bit Raspberry Pi OS as well as AWS Kubernetes Cluster with ARM

determine the requirements and the process to get the operator in
prepare everything mentioned in https://operatorhub.io/contribute
add the operator to https://github.com/k8s-operatorhub/community-operators/tree/main/operators

migrate to mondoohq/mondoo-operator

We are migrating to a new repository:

move the repo itself
update setting and pipeline to work with the new name

Application Scanning

Enable k8s application scanning using mondoo

Use Default Inventory

in #19 we combined the api and node scan into one configuration:

apiVersion: k8s.mondoo.com/v1alpha1
kind: MondooClient
metadata:
  name: mondoo-client
  namespace: mondoo-operator-system
data:
  config: |
    mrn: //agents.api.mondoo.app/spaces/test-infallible-taussig-796596/serviceaccounts/1u20vCfgWqaxOjGmWFFCH4qi2se
    space_mrn: //captain.api.mondoo.app/spaces/test-infallible-taussig-796596
    private_key: |
      -----BEGIN PRIVATE KEY-----
      MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDB4SxLzF7ZQvD0dxoWa
      ...
      pJLAp0cuKBUrUNpOQ62qaQ9F17/r6/TBejq6FaYkok7og+MkQVr8gos=
      -----END PRIVATE KEY-----
    certificate: |
      -----BEGIN CERTIFICATE-----
      MIICfDCCAgGgAwIBAgIRAKqQ9zkDA/rIBj96r4g+qJswCgYIKoZIzj0EAwMwSTFH
      ...
      Rc3OFH5K0IWA0yDdL5QVoQ==
      -----END CERTIFICATE-----
    api_endpoint: https://api.mondoo.app
 
  kubeapi:
    disable: true
    inventory: |
      apiVersion: v1
      kind: Inventory
      metadata:
        name: mondoo-k8s-api-inventory
        labels:
          environment: production
      spec:
        assets:
          - id: api
            connections:
              - backend: k8s
  kubenodes:
    disable: true
    inventory: |
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: mondoo-inventory
      data:
        inventory: |
          apiVersion: v1
          kind: Inventory
          metadata:
            name: mondoo-k8s-inventory
            labels:
              environment: production
          spec:
            assets:
              - id: host
                connections:
                  - host: /mnt/host
                    backend: fs

While this works, it is cumbersome to define the inventory for the user, especially since the inventory stays 99% identical. Therefore we want to make it easy for the users to use the defaults:

apiVersion: k8s.mondoo.com/v1alpha1
kind: MondooClient
metadata:
  name: mondoo-client
  namespace: mondoo-operator-system
data:
  config: |
    ..
  kubeapi:
    disable: false
  daemonset:
    disable: false

If no inventory is provided, we should use the defaults.