Note: This is not an officially supported Google product
The Cluster Bundle is a Kubernetes-related project developed to improve the way we package and release Kubernetes software. It was created by the Google Kubernetes team, and was developed based on our experience managing Kubernetes clusters and applications at scale in both GKE and GKE On-Prem.
It is currently experimental (Pre-Alpha) and will have frequent, breaking changes until the API settles down.
Most users will interact with Cluster Bundle objects via the command line
interface bundlectl.
To install, run:
go install github.com/GoogleCloudPlatform/k8s-cluster-bundle/cmd/bundlectlPackaging in Cluster Bundle revolves around a new type, called the Component:
- Component: A versioned collection of Kubernetes objects. A component should correspond to a logical application. For example, we might imagine components for each of Etcd, Istio, KubeProxy, and CoreDNS.
- ComponentBuilder: An intermediate type that allows for easier creation of component objects.
The Cluster Bundle APIs are minimal and focused on the problem of packaging Kubernetes objects into a single unit. It is assumed that some external deployment mechanisms like a command line interface or an in-cluster controller will consume the components and apply the Kubernetes objects in the component to a cluster.
Here's how you might use the Component type to represent Etcd:
apiVersion: bundle.gke.io/v1alpha1
kind: Component
spec:
# A human readable name for the component.
componentName: etcd-component
# SemVer version for the component representing any and all changes to the
# component or included object manifests. This should not be tied to the
# application version.
version: 30.0.2
# A list of included Kubernetes objects.
objects:
- apiVersion: v1
kind: Pod
metadata:
name: etcd-server
namespace: kube-system
spec:
containers:
- command:
- /bin/sh
- -c
- |
exec /usr/local/bin/etcd
# ... and so onSome notes about components:
- Components have a name and version. The pair of these should be a unique identifier for the component.
- Objects must be Kubernetes objects -- they must have
apiVersion,kind, andmetadata. Other than that, there are no restrictions on what objects can be contained in a component.
You can see more about examples of components in the examples directory.
To make it easier to create components, you can use the ComponentBuilder type:
apiVersion: bundle.gke.io/v1alpha1
kind: ComponentBuilder
componentName: etcd-component
version: 30.0.2
# File-references to kubernetes objects that make up this component.
objectFiles:
- url: file:///etcd-server.yamlComponentBuilders can be built into Components with:
bundlectl build --input-file=my-component.yamlDuring build, objects are inlined, which means they are imported directly
into the component.
Raw-text can also be imported into a component. When inlined, this text is converted into a ConfigMap and then added to the objects list:
apiVersion: bundle.gke.io/v1alpha1
kind: Component
spec:
canonicalName: data-blob
version: 0.1.0
rawTextFiles:
- name: data-blob
files:
- url: file:///some-data.txtThe Cluster Bundle library provides a new type called the PatchTemplate to
perform Kustomize-style
patching to objects in component. These are Go-templates that are applied to
the objects in a component via
StrategicMergePatch
or, if the object is not a Kubernetes Object, via JSONPatch.
For example, if you have the following PatchTemplate specified as an object in a component:
apiVersion: bundle.gke.io/v1alpha1
kind: PatchTemplate
template: |
apiVersion: v1
kind: Pod
metadata:
namespace: {{.namespace}}and the following Kubernetes object in the same component:
apiVersion: v1
kind: Pod
metadata:
name: etcd-server
spec:
containers:
- command:
- /bin/sh
- -c
- |
exec /usr/local/bin/etcd
# etc...and you specified following options YAML file:
namespace: foo-namespaceThen, running the bundlectl command like so:
bundlectl patch --input-file=etcd-component.yaml --options-file=options.yamlproduces:
apiVersion: v1
kind: Pod
metadata:
name: etcd-server
namespace: foo-namespace
spec:
containers:
- command:
- /bin/sh
- -c
- |
exec /usr/local/bin/etcd
# etc...For more examples of Patching and Patch-Building, see the examples directory.
Additionally, you can provide an OpenAPIv3 schema to validate and provide
defaults for options applied to patch templates. Extending the example above,
we can provide defaulting and validation for our namespace parameter.
apiVersion: bundle.gke.io/v1alpha1
kind: PatchTemplate
template: |
apiVersion: v1
kind: Pod
metadata:
namespace: {{.namespace}}
optionsSchema:
properties:
namespace:
type: string
pattern: '^[a-z0-9-]+$'
default: dev-nsHere, we require the namespace parameter to be a string that must match the
regex pattern. Additionally, if the namespace parameter is not supplied, then
we default namespace to dev-ns.
bundlectl can filter objects from a component, which allows for powerful
chaining of the bundlectl command.
# Filter all config map objects
bundlectl filter -f my-component.yaml --filterType=objects --kind=ConfigMap
# Keep only config map objects.
bundlectl filter -f my-component.yaml --filterType=objects --kind=ConfigMap --invert-matchThe Cluster Bundle library provides experimental support for testing
building, patching, and validating components, by writing YAML test-suites that
are run via go test.
To write a component test suite, create a YAML test file like so:
componentFile: etcd-component-builder.yaml
rootDirectory: './'
testCases:
- description: Success
apply:
options:
namespace: default-ns
buildLabel: test-env
expect:
objects:
# The kind + the name references a unique object in the component.
- kind: Pod
name: etcd-server
# ensure that the generated 'etcd-server' object YAML has the following
substrings
findSubstrs:
- 'build-label: test-env'
- 'image: k8s.gcr.io/etcd:3.1.11'
- 'namespace: default-ns'
# ensure that the generated 'etcd-server' object YAML does not have the
# following substrings
notFindSubstrs:
- 'build-label: dev-env'
- description: 'Fail: parameter missing'
expect:
# Check for an error condition during the apply-process.
applyErrSubstr: 'namespace in body is required'Then, you need only write the following go boilerplate:
package component
import (
"testing"
"github.com/GoogleCloudPlatform/k8s-cluster-bundle/pkg/testutil/componentsuite"
)
func TestComponentSuite(t *testing.T) {
componentsuite.Run(t, "test-suite.yaml")
}Running the tests can be performed by running go test.
Examples:
During pre-Alpha and Alpha, breaking changes may happen in any of the types in
pkg/apis, the bundlectl CLI, or the Go APIs (methods, structs, interfaces
not in pkg/apis).
During Beta, backwards incompatible breaking changes in the bundlectl and the
pkg/apis directory will only happy during Major version boundaries.
The Go APIs are not covered by any compatibility guarantee and can break in backwards incompatible ways during any release
This directory follows a layout similar to other Kubernetes projects.
pkg/: Library code.pkg/apis: APIs and schema for the Cluster Bundle.pkg/client: Generated client for the Cluster Bundle types.config/crds: Generated CRDs for the ClusterBundle types.examples/: Examples of componentscmd/: Binaries. This contains thebundlectlCLI tool.
The Cluster Bundle relies on Bazel for building and
testing, but the library is go-gettable and it works equally well to use go build and go test for building and testing.
To run the unit tests, run
bazel test //...It should also work to use the go command:
go test ./...To make using Bazel easier, we use Gazelle to automatically write Build targets. To automatically write the Build targets, run:
bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro 'repositories.bzl%gomod_repositories'We rely heavily on Kubernetes code generation. To regenerate the code, run:
hack/update-codegen.shIf new files are added, you will need to re-run Gazelle (see above).
We use prow to test the Cluster Bundle. If the prow-jobs need to be updated, see the Cluster Bundle Prow Jobs:
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent