Wishlist of CEL libraries about kubernetes HOT 18 OPEN

fwiw, optionals can make that more concise:

self.?status.?conditions.orValue([]).exists(c, c.type == 'QuotaReserved' && c.status == 'True')

from kubernetes.

/sig api-machinery
cc @cici37 @jpbetz

from kubernetes.

@IrvingMg, @trasc did I miss anything important?

from kubernetes.

What do you mean by Pod templates @alculquicondor?

One pattern I'd like to see used more: when you create something that embeds a Pod template, the controller for that kind tries to dry-run make a PodTemplate. That way you get one place to put customer validation (eg a ValidatingAdmissionPolicy), and it can apply to lots of API kinds without repetition.

from kubernetes.

One pattern I'd like to see used more: when you create something that embeds a Pod template, the controller for that kind tries to dry-run make a PodTemplate.

Interesting. I've never seen that. It sounds bullet proof from a validation perspective. Is there dry-run support in the apiserver? But then it would have to be called from the webhook?

from kubernetes.

Regexes for object names, label keys, values, container names, etc. I think this one is already in the works?

Yes, this one is progressing here: #123572 (cc @alexzielenski)

from kubernetes.

The ultimate validation: Pod templates, but worth starting with just containers :) Very useful for job CRDs.

We might do something special to validated embedded types like this that doesn't involved CEL. But yes, I agree there is a huge need here. Do you happen to have any references to specific use cases? I'm working on accumulating those.

from kubernetes.

Is there dry-run support in the apiserver?

yes, since 1.12: https://github.com/kubernetes/kubernetes/blob/release-1.12/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go#L485-L491

from kubernetes.

Re: dry-run support
I see. Still, that would imply that a webhook has to do an API call. Would you still recommend this?

Do you happen to have any references to specific use cases?

1. All of the kubeflow job CRDs, to start. They will eventually fail at Pod or k8s Job creation, if the template is wrong, as they are not doing any validation.
2. Kueue Workload objects https://kueue.sigs.k8s.io/docs/concepts/workload/. These are created out of existing Jobs, Pods, or arbitrary CRDs (like Kubeflow jobs). The only problematic case is the last one.

from kubernetes.

3. JobSet validation (based on JobTemplates) kubernetes-sigs/jobset#422

from kubernetes.

@danielvegamyhre was looking into kubectl-validate as a way to validate templates as a library.

from kubernetes.

@IrvingMg, @trasc did I miss anything important?

Not that I can think of. That's everything we need for now.

from kubernetes.

WRT conditions, right now, we have to do things like this:

has(self.status) && has(self.status.conditions) && self.status.conditions.exists(c, c.type == 'QuotaReserved' && c.status == 'True')

It would be good to have a simplified experience, similar to meta.IsConditionTrue in golang.

I added a separate item for this.

from kubernetes.

/cc @cici37 @alexzielenski @jpbetz
/triage accepted

from kubernetes.

@alculquicondor is part of the issue that there isn't support for variables within CRD validations? I'm certain that the ValidatingAdmissionPolicy support for variables is instrumental in making the config as DRY as possible.

from kubernetes.

FWIW I have a draft KEP I was hoping to implement this release (maybe deferred to next) to add variables also to CRDs

kubernetes/enhancements#4590

from kubernetes.

I wasn't aware that CEL itself supported variables. That could help.

But, in general, there are common structs that multiple APIs might want to use, and we should have library validations for those.

from kubernetes.

But, in general, there are common structs that multiple APIs might want to use, and we should have library validations for those.

100% agree on this. I'd like it to feel to a CEL user like the language "understand" kubernetes resources and the types found within them. This includes quantities, durations, date-times, int-or-string, IPs, CIDRs, and all the name formats,. and maybe more sophisticated types like Conditions, selectors... We're have support for many of these and are actively working on some others, but we definitely have gaps.

from kubernetes.