tazjin / kontemplate Goto Github PK

Title says it all ...

This implicitly fixes the first point mentioned in #76

This is allowed with regular kubectl:

$ kubectl apply -f path/to/file-deploy.yaml

These don't seem to work:

$ kontemplate template beta-cluster.yaml -i path/to/file-deploy.yaml
 │Loading resources for path/to/file-deploy.yaml 

$ kontemplate template beta-cluster.yaml -i path/to/file-deploy
 │Loading resources for path/to/file-deploy

Meaning, they don't output the rendered file.

I attempted with the following beta-cluster.yaml:

---
context: lala
include:
  - name: path/to
    values:
      lala: land

  - name: path/to/file-deploy
    values:
      configStuff: stuff

  - name: path/to/file-deploy.yaml
    values:
      configStuff: stuff

Setting missingkey=error breaks the default function in sprig.

Default values are somewhat of a requirement though, so I'm thinking this influences #28 towards option one by necessitating #25

Now that #6 is cancelled I'm bouncing around other ideas for solving the CI issue.

Problem

In an automated CI system we want the versions (i.e. image tags) of artifacts to change after new images have been created for software. Somehow this change needs to propagate to kontemplate.

Potential solutions

There are two approaches that can be taken:

1. Resource set default values

Implement #25 to allow resource sets to contain certain default values within them (as a values.json which is easy to edit programmatically). This could be implemented with relatively few code changes and would enable CI systems to update the default versions in resource sets after new images have been deployed.

This works well for "always master"-style deployments, and versions could still be pinned by overriding them in the context. If the context doesn't specify any version then whatever the default is in the resource set will be used.

Example:

# some-api/deployment.yaml
kind: Deployment
# [...]
spec:
  template:
    spec:
      containers:
        - image: myrepo/some-api:{{ someApiVersion }}
          # [...]

// some-api/values.json
{
    "someApiVersion": "1.2.3"
}

# somecontext.yaml
context: somecontext
include:
  - name: some-api

This would deploy some-api with whatever version is specified and it could be overridden in the context (if necessary). This way automatic deploys could be "disabled" for certain artifacts if a CI system applies all environments constantly.

2. Make versions a first-class citizen

In this model, versions could become a first-class field of the ResourceSet type that is optionally loaded in from a different file. This file would be in JSON format and easy to edit programmatically.

Example:

# somecontext.yaml
context: somecontext
versions: somecontext-versions.json
include:
  - name: some-api

// somecontext-versions.json
{
  "some-api": "1.2.3"
}

This would set the version field in that parsed resource set to "1.2.3".

CI workflows

Ideally a CI system could run kontemplate apply -f context.yaml without any further options, i.e. without filtering for certain resource sets, after every change to the repository in which kontemplate configuration is kept.

For the solutions mentioned above the workflows look slightly, but not much, different:

1. After some-api is built, a subsequent job creates a commit in the kontemplate-config repo that changes the values.json file of the some-api resource set.

2. After some-api is built, a subsequent job creates a commit in the kontemplate-config repo that updates the somecontext-versions.json file.

The updated kontemplate-config repo triggers a CI run of kontemplate apply.

Pros & Cons

Intuitively I think that version 1 leads to a nicer CI flow as there aren't multiple files that need to be updated and things are sort of self-contained. Somehow when written down version 2 seems simpler though.

See title, not much more to say!

It should be possible to toggle and exclude certain resource sets from runs that use kubectl apply.

Some Kubernetes resources will break kubectl if you attempt to apply them, such as any instance of a ThirdPartyResource or newer resources like PodDisruptionBudget.

It looks like this may be fixed in kubernetes/kubernetes#40666 but either way that won't be released for a while longer.

Third-party resources don't work with apply and this causes pain.

Currently a typo in an -i flag will just result in a noop.

Seems camelCasing / PascalCasing / snake_casing is supported but not lisp-case(hello_world). Would be nice if it said in the documentation what casing is supported and preferably throw an explanatory error when user uses lisp-case.

There is an err check missing 🙄

I would really appreciate some hints about integrating kontemplate in a GitLab CI pipeline. Since you already such a setup up-n-running, are you able to give a rough outline of what's needed?

Windows can't figure out that it's executable otherwise!

Related to a discussion about #53, @heavenlyhash brought up that all I/O should be toggleable via flags.

Current I/O methods:

• passLookup function calls pass command line tool
• getResource calls kubectl to retrieve resource configuration (#61)
• (NYI) resource sets can be retrieved from external locations (#53)
• (NYI) arbitrary files from resource sets can be loaded and templated (#46)

Should at least have tests for certain things in templater, such as resource set inclusion/exclusion.

Title says it all.

For slightly better organisational structure. Assuming for example:

.
├── controllers
│   ├── flink-controller
│   └── letsencrypt-controller
├── test-cluster.yaml

It should be possible to write in test-cluster.yaml:

include:
  - name: controllers
     including:
      - name: flink-controller
      - name: letsencrypt-controller

(syntax TBD)

Opposite of --limit

Right now it's possible to do {{ .someValue | default "something" }}, however it may be of interest to be able to put a default values.(json|yaml) file into a resource set.

Due to how difficult #6 is looking to be it would be great to simply be able to use jq for this in the first round.

As pointed out by @mazzus, this is currently a bug on master.

more information here soon

I like the ideas behind this project, but I struggled to get it working with my existing setup. Do with these comments what you will, I just wanted to give some constructive feedback. Some problems I observed:

No errors are reported if a file wasn't found

There is just no output. I spent a while figuring out why there was no output from kontemplate template, even though everything looked fine. Turns out the path was incorrect.

The path to a "service folder" silently fails if it is a file instead of a directory

I assumed that kontemplate ... --include foo/bar/baz.yml would work, but it doesn't. It will try to silently load all files below the non-existent directory foo/bar/baz.yml, which obviously fails. I don't actually mind this being unspported, but it would be nice to get an error.

Paths are relative to the context yml files

I expected paths to be relative to the current working directory, but apparently they are relative to the yml files that contain the context definitions. If the context yml file are in a subdirectory you'll get commands like kontemplate template config/environments/staging.yml --include ../../apps/site-static. This reads super weird if apps/site-static actually exists in the current directory.

Package.json files are included by default

These are commonly used for Node projects and it would be nice if they were excluded by default, or if there were a way for me to manually exclude them.

If someone forgets a -i | --include Kontemplate should fail.

Updates in sprig are causing templates to be inconsistent if users build kontemplate on their own machines.

Versions must be pinned now!

It should be possible to fetch secrets from pass using a template function.

Most of my kontemplate-based setups are using Jenkins so far, so it makes sense to extract the common jobs as a pipeline library.

There are basically two main jobs:

• kontemplateRun: This job is executed as a build pipeline for the repository containing kontemplate resuorce sets and cluster configurations. It defaults to apply, has a parameter for the cluster context/config to use and a parameter for toggling --dry-run.
  This job should have a (configurable) default context and a visual drop-down in the Jenkins GUI for running on other contexts.
  It should be useable with --dry-run on pull requests/WIP changes to a kontemplate repo, automatic deploys of test clusters on merge etc.

• resourceSetVersionBump: (this may need a better name!) This job can be parameterised with the name/path for a resource set, a version to set and (optionally) the version parameter key. It should then be able to go and update a default.json in the kontemplate repository to enable a version bump flow as described in #28

These jobs should expect kontemplate, kubectl and jq (for the second one) to be available on the build slave. Maybe a preconfigured image can be optionally provided for Jenkins setups that are running in k8s, but that is not a priority.

People may expect this to happen! Right now they will just get discarded.

https://github.com/tazjin/kontemplate/blob/master/main.go#L23

It would be really nice to have the option to include external resources via HTTP so that I could have templates stored somewhere different than the kontemplate file.

Use case 1

I create a microservice deployment template that I want multiple projects to be able to use. Each project then has one or more kontemplate files that refer to the common deployment template.

Each project only has kontemplate files like:

• dev.yml:

---
context: myservice-dev
include:
- name: https://raw.githubusercontent.com/standard/template/master/service.yaml
  values:
    serviceName: myservice
    servicePort: 8080
    ingressDomain: myservice.dev.mydomain.com

• prod.yml:

---
context: myservice
include:
- name: https://raw.githubusercontent.com/standard/template/master/service.yaml
  values:
    serviceName: myservice
    servicePort: 8080
    ingressDomain: myservice.mydomain.com

Use case 2

Using kontemplate for cluster standup I have a collection of addons that I want configured per cluster. Some addon definitions will be locally managed, some will be shared (and may not even need customization for installation).

dev-cluster.yml:

---
context: k8s.dev.mydomain.com
global:
  clusterDomain: k8s.dev.mydomain.com
include:
- name: addons/ingress
  values:
    name: nginx-ingress
    replicas: 2
    ingressClass: nginx
    ingressDomain: 'k8s.dev.mydomain.com'
- name: addons/ingress
  values:
    name: nginx-ingress-internal
    replicas: 2
    ingressClass: nginx-internal
    ingressDomain: 'internal.k8s.dev.mydomain.com'
- name: addons/logging
  values:
    kibanaIngressClass: nginx-internal
    kibanaDomain: 'kibana.internal.k8s.dev.mydomain.com'
- name: https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.6.0.yaml

prod-cluster.yml:

---
context: k8s.mydomain.com
global:
  clusterDomain: k8s.mydomain.com
include:
- name: addons/ingress
  values:
    name: nginx-ingress
    replicas: 5
    ingressClass: nginx
    ingressDomain: 'k8s.mydomain.com'
- name: addons/ingress
  values:
    name: nginx-ingress-internal
    replicas: 3
    ingressClass: nginx-internal
    ingressDomain: 'internal.k8s.mydomain.com'
- name: addons/logging
  values:
    kibanaIngressClass: nginx-internal
    kibanaDomain: 'kibana.internal.k8s.mydomain.com'
- name: https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.6.0.yaml

Yes for the second use case I could just directly run kubectl apply -f with the URL, but keeping it all in one place would be very convenient and make me happy :)

Rather than just outputting the resources, kontemplate should be able to call kubectl apply directly. Maybe the k8s Go libs can also do this right in the app without shelling out (only if it's not too complicated).

Should have a simple {{ fileContent "foo.txt" }} basically.

The files should always refer to the resource set directory and not allow changing paths.

Could potentially also have {{ templateContent "foo.txt.tpl" }} or something to interpolate ~ deeper ~.

Should be possible to say kontemplate some-cluster.json --limit some-api,other-thing

Title says it all.

An idea that struck me and that may be silly!

It could be interesting to have a template function like resourceHash :: ResourceType -> ResourceName -> Hash (e.g. {{ resourceHash "ConfigMap" "app-config" }}).

This could be used to detect changes to resources that should trigger rolling-updates and such where Kubernetes does not currently provide native support for doing that.

Example use-case:

Someone using the letsencrypt-controller gets a new Secret after the controller refreshed the certificate before it expires.

Some backend-service or Nginx deployment should now be rolling updated to pick up the new certificate. With this resourceHash function the user could write something like this in kontemplate:

annotations:
  secretHash: {{ resourceHash "Secret" "my-website-tls" }}

and on the next run it would be updated.

Implementation thoughts

Under the hood this should basically do a kubectl get --export -o json --context $context <resourcetype> <name> | sha256sum.

I think this may be useful but haven't quite figured it out yet. Keeping this issue around so I don't forget.

Caveats

• Does not happen immediately after resources change (fine for letsencrypt-certs for example as there is a certain grace period with my controller)
• Does not work in template mode (in template mode we can not call kubectl, that would be against the rules)

I recreated the example from the docs with forwarder-asia etc. after I noticed path not seeming to work, and even that example doesn't seem to work (unless I'm missing something)

I've got my forwarders.yml copy/pasted from the example, i.e.:

include:
  - name: forwarder-europe
    path: tools/forwarder
    values:
      source: europe
  - name: forwarder-asia
    path: tools/forwarder
    values:
      source: asia

My directory structure is:

test
├── forwarders.yml
└── tools
    └── forwarder
        └── forwarder.yml

And just to prove the variable substitution works, my tools/forwarder/forwarder.yml is just:

source: {{ .source }}

But, I (sliently) get no output when I run things through kontemplate template like so:

$ kontemplate template test/forwarders.yml 
Loading resources for forwarder-europe
Loading resources for forwarder-asia

If I change that top-level forwarders.yml from the example to remove the path directive and just use the same name for both like so:

include:
  - name: tools/forwarder
    values:
      source: europe
  - name: tools/forwarder
    values:
      source: asia

I get the output I would originally have expected from the example:

$ kontemplate template test/forwarders.yml 
Loading resources for tools/forwarder
Loading resources for tools/forwarder
Rendered file tools/forwarder/forwarder.yml:
source: europe
Rendered file tools/forwarder/forwarder.yml:
source: asia

I've tried several different configurations with this, and they all seem to point back to the path directive just being ignored.

Otherwise Jenkins may do funny things such as not erroring on failures when kontemplate is used in CI.

Obviously this comes up shortly after stable release \o/

Kontemplate seems to accept the .yml extension for most files, but not for the files used to specify default values.

I don't really like YAML, but other people may want it.

Currently the path from which a resource set's files are loaded is inferred from the name of the resource set.

In some situations, such as importing the same resource set multiple times with different configuration, the user may want to split paths and names. This should be supported:

- name: api-europe
  path: some-api
  values:
    location: europe
- name: api-asia
  path: some-api
  values:
    location: asia

thus making it possible to use --include / --exclude api-asia for example.

Should feature docs on:

• better usage example
• resource set collections
• notes about things like include/exclude precedence

kubectl apply accepts yaml documents with multiple docs without --- in head.
But without this seperator in head of each template the generated template from kontemplate becomes invalid.

A Deployment can have a kubernetes.io/change-cause annotation that will show up in kubectl rollout history.

Kontemplate should support some sort of easy way to insert this sanely, e.g.:

kind: Deployment
metadata:
  annotations:
    {{ .kontemplateChangeCause }}

to expand (intelligently) to something like

kind: Deployment
metadata:
  annotations:
    kubernetes.io/change-cause: kontemplate-v1.1 w/ rev-${gitHashOfConfigRepo}

(just making up a format here but you get the gist of it)

I really like the idea and simplicity of this project but one thing that seems to be missing in my opitinon is to be able to set variable hierarchical (for sharing between environments). So I might have a cluster.yaml file which has:

---
global:
  globalVar: lizards
include:
  - name: some-api
    values:
      version: 1.0-0e6884d
      importantFeature: true
      apiPort: 4567

and then prod-cluster.yaml somehow inherits from the base file. So it gets all defined variable from cluster.yaml and can override them if wanted, e.g.:

---
include:
  - name: some-api
    values:
      importantFeature: false
      someOtherVar: something

This will take the version, globalVar and apiPort from cluster.yaml, override importantFeature and add someOtherVar.

Though the open question is on how to achieve that the best way. Maybe through some import feature? Or placing the files in a hierarchical way on the file system?

So e.g.

.
├── cluster.yaml
└── dev
    ├── cluster.yaml
└── staging
    ├── cluster.yaml
└── prod
    ├── cluster.yaml
└── some-api
    ├── deployment.yaml
    └── service.yaml

Those are just some ideas on how to implement it. What do you think?

This won't make it into today's release, needs some more fleshing out.

This relates to #76 and #69

Should be able to kontemplate set some-api.version=whatever -f some-cluster.json to update the JSON.

Old issue text below the line!

I can't defend (mostly to myself) maintaining a Go project anymore and it's time to go for the Rust rewrite.

Milestone 6 will track the progress of the rewrite. Some choices that have been made:

• Compatibility with existing configuration files will be kept
• Compatibility with existing template files will not be kept
• Templating will not be restricted to just JSON/YAML files

This issue stays open until the milestone is resolved.

Preferably Rust.

Either way, I'd like to not break compatibility with existing setups which means I have to re-implement Go's templates in whatever I choose.

Doing this in Rust could contribute some useful things to the Rust ecosystem I suppose.

Note: This is mostly for my own sanity.

Documentation is a hard problem. How do you make features discoverable to users when they need them?

Currently this isn't solved well in the documentation that the repository has, but I'm thinking that a revamp of the README (which is probably the documentation entrypoint for most people) can partially solve this.

One idea I had was to start off the README with a list of features, each of which are a link to a more specific documentation piece for that feature. For example:

Kontemplate is [... blabla ...]. Check out an example and the feature list below:

[simple usage example]

## Features:

* [Simple, yet powerful templates](docs/templates.md)
* [Clean cluster configuration files](docs/cluster-config.md)
* [Integration with `pass`](docs/pass.md)
* [Integration with `kubectl`](docs/cli.md)
* ... other stuff

## Best practices

Kontemplate is being used in several real-world Kubernetes deployments.

Check out the best practice documentation on [code organisation](docs/code-organisation.md) and [CI integration](docs/ci-integration.md).

The one thing that is still missing here is something that communicates the term resource set clearly to the user. Ideally there would also be a link to an example project that makes use of as many features as possible (*sigh* I should update all my tazj.in things).