reevaluate CLI design about kubenix HOT 10 CLOSED

One drawback is that kubectl diff isn't as smart as helm diff in that it will throw an error if any resources reference missing namespaces or CRDs (even if said resources are within the same set). Not a deal breaker, I don't feel.

from kubenix.

Will also need a way to filter out noise if we're going to have useful diffs; otherwise, there will be countless

-    kubenix/hash: bf784eba18bc4ac54f30ad01873205d7486b0163
+    kubenix/hash: 67c3651392deea6752d5d05c0e83a369e6caabaa

on every execution.

from kubenix.

Hashes cluttering up the diff was solved by using KUBECTL_EXTERNAL_DIFF to add the -I flag for ignoring patterns like so:

diff -N -u -I ' kubenix/hash: ' $@

So the linked MR for the full implementation.

from kubenix.

I've added basic deletion with --prune; which seems to work fairly well for my simple use-case. 1.27 adds support for ApplySets which appears to be an even more robust approach to avoid adding said logic here.

More importantly, that raises another issue of verifying/matching the version skew policy. Should be possible since we already have access to the user's API version.

from kubenix.

access to rendered manifests before they're applied; this is necessary to, e.g., inject secrets

For what it's worth, the approach I use it to keep encrypted secrets within the repo and the k8s manifests and have them decrypted in an init container (a slightly evolved version of this), before the pod starts. I use sops for encrypting\decrypting as it integrates with various cloud providers.

generate a diff prior to apply

I use an intermediate repo with final generated manifests and then argocd which watches for changes on the repo to syncs them to the cluster. This also takes care of pruning deleted resources.

So a CI job, instead of running kubernetes apply, simply generates the manifests and pushes them to the manifest repo, either directly to main branch or creating a PR first which can be reviewed and acts as gating mechanism for deploys. This also allows you to make you cluster fully private and not accessible from the outside world (as argocd runs within the cluster).

from kubenix.

For what it's worth, the approach I use it to keep encrypted secrets within the repo and the k8s manifests and have them decrypted in an init container (a slightly evolved version of this), before the pod starts.

I like agenix as it matches how I manage secrets outside of k8s but this is also a nice approach (that's probably worth adding to the docs). I might have to play around with something along these lines and see how it goes.

I use an intermediate repo with final generated manifests and then argocd which watches for changes on the repo to syncs them to the cluster. This also takes care of pruning deleted resources.

I think gitops is a bit overkill for my own use-case. ArgoCD is pretty nice but adds more complexity than I personally want/need. That said, it's also a perfectly valid approach and does solve a lot of the same problems in a slightly different way.

I appreciate having these details though. I'd like to beef up the docs with more stuff like this. I recently added a "tips-n-tricks" section which I mean to use as a bucket to collect pages for these sort of "this is how you might do X" approaches. Might not be the best title but having them is nicer than having to discover things on your own.

from kubenix.

ArgoCD is pretty nice but adds more complexity than I personally want/need.

Fair point.

I appreciate having these details though. I'd like to beef up the docs with more stuff like this. I recently added a "tips-n-tricks" section

Cool, happy to contribute

from kubenix.

Another tool which can be used to track changes and prune resources when deploying kubectl apply style: https://carvel.dev/kapp/

from kubenix.

Ooh, kapp looks to fit this definition pretty nicely with ordered applies, ready checks, even targeting with labels. Thanks for pointing that out, hadn't seen it before.

So maybe a bigger question is should we provide a "golden" apply path? The alternative (so far as I understand) is just documenting a few suggestions and leaving implementation to the user.

from kubenix.

Thanks for being a sounding board here, @adrian-gierakowski 💯 I'm going to assume most users are in the same boat as you and are currently relying on some outside deployment mechanism (which is perfectly fine, and in many ways better, of course; maybe I'll go that way myself one of these days 🙃).

I've merged the associated MR which does about all I'm personally looking for here (will continue to iterate, of course).
I'm going to open a new issue for myself to document some of these alternative deployment methods based your feedback.

from kubenix.