Giter Club home page Giter Club logo

kubewebhook's Introduction

kubewebhook

kubewebhook

CI Go Report Card GoDoc Apache 2 licensed GitHub release (latest SemVer) Kubernetes release

Kubewebhook is a small Go framework to create external admission webhooks for Kubernetes.

With Kubewebhook you can make validating and mutating webhooks in any version, fast, easy, and focusing mainly on the domain logic of the webhook itself.

Features

  • Ready for mutating and validating webhook kinds.
  • Abstracts webhook versioning (compatible with v1beta1 and v1).
  • Resource inference (compatible with CRDs and fallbacks to Unstructured).
  • Easy and testable API.
  • Simple, extensible and flexible.
  • Multiple webhooks on the same server.
  • Webhook metrics (RED) for Prometheus with Grafana dashboard included.
  • Webhook tracing with Opentelemetry support.
  • Supports warnings.

Getting started

Use github.com/slok/kubewebhook/v2 to import Kubewebhook v2.

func run() error {
    logger := &kwhlog.Std{Debug: true}

    // Create our mutator
    mt := kwhmutating.MutatorFunc(func(_ context.Context, _ *kwhmodel.AdmissionReview, obj metav1.Object) (*kwhmutating.MutatorResult, error) {
        pod, ok := obj.(*corev1.Pod)
        if !ok {
            return &kwhmutating.MutatorResult{}, nil
        }

        // Mutate our object with the required annotations.
        if pod.Annotations == nil {
            pod.Annotations = make(map[string]string)
        }
        pod.Annotations["mutated"] = "true"
        pod.Annotations["mutator"] = "pod-annotate"

        return &kwhmutating.MutatorResult{MutatedObject: pod}, nil
    })

    // Create webhook.
    wh, err := kwhmutating.NewWebhook(kwhmutating.WebhookConfig{
        ID:      "pod-annotate",
        Mutator: mt,
        Logger:  logger,
    })
    if err != nil {
        return fmt.Errorf("error creating webhook: %w", err)
    }

    // Get HTTP handler from webhook.
    whHandler, err := kwhhttp.HandlerFor(kwhhttp.HandlerConfig{Webhook: wh, Logger: logger})
    if err != nil {
        return fmt.Errorf("error creating webhook handler: %w", err)
    }

    // Serve.
    logger.Infof("Listening on :8080")
    err = http.ListenAndServeTLS(":8080", cfg.certFile, cfg.keyFile, whHandler)
    if err != nil {
        return fmt.Errorf("error serving webhook: %w", err)
    }

    return nil

You can get more examples in here

Production ready example

This repository is a production ready webhook app: https://github.com/slok/k8s-webhook-example

It shows, different webhook use cases, app structure, testing domain logic, kubewebhook use case, how to deploy...

Static and dynamic webhooks

We have 2 kinds of webhooks:

  • Static: Common one, is a single resource type webhook.
  • Dynamic: Used when the same webhook act on multiple types, unknown types and/or is used for generic stuff (e.g labels).
    • To use this kind of webhook, don't set the type on the configuration or set to nil.
    • If a request for an unknown type is not known by the webhook libraries, it will fallback to runtime.Unstructured object type.
    • Very useful to manipulate multiple resources on the same webhook (e.g Deployments, Statefulsets).
    • CRDs are unknown types so they will fallback to runtime.Unstructured`.
    • If using CRDs, better use Static webhooks.
    • Very useful to maniputale any metadata based validation or mutations (e.g Labels, annotations...)

Compatibility matrix

To know the validated compatibility, check the integration tests on CI.

Kubewebhook Kubernetes Admission reviews Dynamic webhooks OpenTelemetry tracing
v2.6 1.29, 1.28, 1.27, 1.26 v1beta1, v1
v2.5 1.25 v1beta1, v1
v2.4 1.24 v1beta1, v1
v2.3 1.23 v1beta1, v1
v2.2 1.22 v1beta1, v1
v2.1 1.21 v1beta1, v1
v2.0 1.20 v1beta1, v1
v0.11 1.19 v1beta1
v0.10 1.18 v1beta1
v0.9 1.18 v1beta1
v0.8 1.17 v1beta1
v0.7 1.16 v1beta1
v0.6 1.15 v1beta1
v0.5 1.14 v1beta1
v0.4 1.13 v1beta1
v0.3 1.12 v1beta1
v0.2 1.11 v1beta1
v0.2 1.10 v1beta1

Documentation

You can access here.

kubewebhook's People

Contributors

al-cheb avatar annymsmthd avatar bonifaido avatar dependabot[bot] avatar eastwd avatar frimidan avatar gecube avatar jesusvazquez avatar luodw avatar opetrushka-zendesk avatar paalka avatar slok avatar soorena776 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kubewebhook's Issues

can not decode pod object when deleting pod

I am using k8s 14 version, and set up a delete pod webhook. By using v10.0 kubewebhook, but getting a empty pod object like:

Pod{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:PodSpec{Volumes:[]Volume{},Containers:[]Container{},RestartPolicy:,TerminationGracePeriodSeconds:nil,ActiveDeadlineSeconds:nil,DNSPolicy:,NodeSelector:map[string]string{},ServiceAccountName:,DeprecatedServiceAccount:,NodeName:,HostNetwork:false,HostPID:false,HostIPC:false,SecurityContext:nil,ImagePullSecrets:[]LocalObjectReference{},Hostname:,Subdomain:,Affinity:nil,SchedulerName:,InitContainers:[]Container{},AutomountServiceAccountToken:nil,Tolerations:[]Toleration{},HostAliases:[]HostAlias{},PriorityClassName:,Priority:nil,DNSConfig:nil,ShareProcessNamespace:nil,ReadinessGates:[]PodReadinessGate{},RuntimeClassName:nil,EnableServiceLinks:nil,PreemptionPolicy:nil,Overhead:ResourceList{},TopologySpreadConstraints:[]TopologySpreadConstraint{},EphemeralContainers:[]EphemeralContainer{},},Status:PodStatus{Phase:,Conditions:[]PodCondition{},Message:,Reason:,HostIP:,PodIP:,StartTime:<nil>,ContainerStatuses:[]ContainerStatus{},QOSClass:,InitContainerStatuses:[]ContainerStatus{},NominatedNodeName:,PodIPs:[]PodIP{},EphemeralContainerStatuses:[]ContainerStatus{},},}

fix jsonpath create wrong patch

Hi, I have use kubewebhook to create k8s webhook service to do crd resource defaults and validates. I found a bug in mutating webhook service when create json patch in https://github.com/slok/kubewebhook/blob/master/pkg/webhook/mutating/webhook.go#L147.

It does not create json patch as expected, because it use origJSON and mutatedJSON to create json patch. The right way to create json patch is using AdmissionReview.Request.Object.Raw and mutatedJSON, the reason is follows:

  1. The origJSON may be different from AdmissionReview.Request.Object.Raw,because AdmissionReview.Request.Object.Raw -->unmarshaled to --> origOBJ --> marshaled to --> origJSON, so if AdmissionReview.Request.Object.Raw has some attr missing,origJSON will has a init value;
  2. As some attrs, origJSON has init value but AdmissionReview.Request.Object.Raw has none , so when create json patch, it will has wrong operation, maybe add, but replace;

kubernetes-sigs/controller-runtime also use AdmissionReview.Request.Object.Raw to create json patch
https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/webhook/admission/defaulter.go#L74

Error message should be returned directly as plaintext upon error

When an error is returned, kubewebhook returns error 500 with the JSON-encoded AdmissionReview response object. However, Kubernetes doesn't actually decode this response body, it just displays an error like this when failurePolicy: Fail:

Error from server (InternalError): Internal error occurred: failed calling webhook "example.com": an error on the server ("{\"kind\":\"AdmissionReview\",\"apiVersion\":\"admission.k8s.io/v1\",\"response\":{\"uid\":\"00000000-0000-0000-0000-000000000000\",\"allowed\":false,\"status\":{\"metadata\":{},\"status\":\"Failure\",\"message\":\"validator error: {actual error string from user}\"}}}") has prevented the request from succeeding

(and when failurePolicy: Ignore, it assumes the webhook is allowed, which is expected)

This is a bit misleading to users. A hurried new user to webhooks may skim through the Kubernetes docs on webhooks and see that an AdmissionReview response should be returned, and then assume that returning an error results in a rejection based on how the above error message looks like (because it resembles an actual rejection).

Why do we need to encode the error into a JSON object when the apiserver doesn't really handle it?

Go version in examples/Dockerfile should be updated


Go version in examples/Dockerfile should be updated to at least 1.13
https://github.com/slok/kubewebhook/blob/master/examples/Dockerfile#L1

The version has been updated in most other places in e1705517587ca08b8c0dd2e39d7e25cf1d092c12


Trying to build as is results in the error:


Step 5/9 : RUN CGO_ENABLED=0 go build -o /bin/example --ldflags "-w -extldflags '-static'"  github.com/slok/kubewebhook/examples/${example}
 ---> Running in 7a05851f052e
# github.com/slok/kubewebhook/vendor/k8s.io/apimachinery/pkg/util/errors
vendor/k8s.io/apimachinery/pkg/util/errors/errors.go:99:10: undefined: errors.Is

# github.com/slok/kubewebhook/vendor/sigs.k8s.io/structured-merge-diff/v3/value
vendor/sigs.k8s.io/structured-merge-diff/v3/value/mapreflect.go:95:13: val.MapRange undefined (type reflect.Value has no field or method MapRange)
vendor/sigs.k8s.io/structured-merge-diff/v3/value/mapreflect.go:173:14: rhs.MapRange undefined (type reflect.Value has no field or method MapRange)
vendor/sigs.k8s.io/structured-merge-diff/v3/value/mapreflect.go:194:13: lhs.MapRange undefined (type reflect.Value has no field or method MapRange)
The command '/bin/sh -c CGO_ENABLED=0 go build -o /bin/example --ldflags "-w -extldflags '-static'"  github.com/slok/kubewebhook/examples/${example}' returned a non-zero code: 2
make: *** [build-examples] Error 2

Same webhook for different kinds of resource

Currently, WebhookConfig requires a specific object to be passed, like the following:

mcfg := mutatingwh.WebhookConfig{
		Name: "podAnnotate",
		Obj:  &corev1.Pod{},
	}

On the other hand, the MutatingWebhookConfiguration Kubernetes' configuration allows to register the same webhook for different kinds of resource, for example:

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: webhook
  labels:
    app: webhook
    kind: mutator
webhooks:
  - name: webhook.example.net
    clientConfig:
      service:
        name: webhook
        namespace: default
        path: "/mutating"
    rules:
      - operations: [ "CREATE", "UPDATE" ]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["pods", "deployments", "replicationcontrollers", "replicasets", "daemonsets", "statefulsets", "jobs", "cronjobs"]

Is it possibile to configure kubewebhook in a way that all resource can be handled by the same webhook?

Looking into the code, it seems not possible (I might be wrong) because WebhookConfig.Obj is used to de-serialize the object, so it works only with objects of the same kind. IMHO, it's useful to have just one webhook endpoint that can review different kinds of resources.

Let me know, if that is possible in someway.

Thanks in advance!

Check if request is dry run

Hi

I am using the example for multiwebhook as a starting point and in the Mutate function (in file examples/multiwebhook/pkg/webhook/mutating/mutator.go) i added the following code at line 26 :

if whcontext.IsAdmissionRequestDryRun(ctx) {
                 m.logger.Infof("Dry Run")
		return false, nil
	}

I tested by kubectl apply --dry-run test.yaml, but this code doesnt seem to do print the info message .

Is the above a correct way to check whether dry run or not?

to validating the DELETE POD failed

hi, I want to delete pod safely.

so I set a validating web hook, I have to try the kubewebhook.

however, the framework can't check the delete operation. basecase of the delete does not have a body only have a request.

the code of deserializer doesn't take the request to the upper level.

	// Get the object.
	_, _, err := w.deserializer.Decode(ar.Request.Object.Raw, nil, runtimeObj)
	if err != nil {
		err = fmt.Errorf("error deseralizing request raw object: %s", err)
		return w.toAdmissionErrorResponse(ar, err)
	}

about CRD

I need to control the access of CRD use mutatewebhook. It seems that this framework will not work,does it support CRD?

apiversion and kind not provide in response

error msg:
expected webhook response of admission.k8s.io/v1, Kind=AdmissionReview, got /, Kind=;

response:
{"response":{"uid":"a7aa4dca-6f37-4e74-9041-c7de0a120c23","allowed":true,"patch":"W3sib3AiOiJhZGQiLCJwYXRoIjoiL21ldGFkYXRhL2Fubm90YXRpb25zL211dGF0ZWQiLCJ2YWx1ZSI6InRydWUifSx7Im9wIjoiYWRkIiwicGF0aCI6Ii9tZXRhZGF0YS9hbm5vdGF0aW9ucy9tdXRhdG9yIiwidmFsdWUiOiJwb2QtYW5ub3RhdGUifV0=","patchType":"JSONPatch"}}

my k8s version:
v1.17.2

kubernetes v1.21.3 tls error

The tls error is displayed when the k8s version is v1.21.3, but it is displayed normally when the k8s version is v1.17.12

The error is:
remote error: tls: bad certificate

Asserting inferred type?

I'm getting a weird results in a mutating webhook writtten to process both v1 and v1beta Ingress.
Full code https://github.com/agilestacks/tls-host-controller/blob/master/cmd/tls-host-controller/main.go

import (
	netv1 "k8s.io/api/networking/v1"
	netv1beta1 "k8s.io/api/networking/v1beta1"
)

mt := mutating.MutatorFunc(func(_ context.Context, _ *kwhmodel.AdmissionReview, obj metav1.Object) (*mutating.MutatorResult, error) {
	ingressv1, v1 := obj.(*netv1.Ingress)
	ingressv1beta1, v1beta1 := obj.(*netv1beta1.Ingress)
	if !v1 && !v1beta1 {
		logger.Warningf("v1: %+v", ingressv1)
		logger.Warningf("v1beta1: %+v", ingressv1beta1)
		logger.Warningf("unsupported object kind %s: %+v", reflect.TypeOf(obj), obj)
		return &mutating.MutatorResult{}, nil
	}

If mutating.WebhookConfig{Obj: &netv1beta1.Ingress{}} is set, then it works. If not, then the output is rather cryptic:

v1: nil
v1beta1: nil
unsupported object kind *v1beta1.Ingress: &Ingress{ObjectMeta:{prometheus-operator-grafana  monitoring  8fbdff6f-a83c-44e5-bb15-ed7f999571ba 19038466 3 2021-02-10 15:54:25 +0000 UTC <nil> <nil> map[app.kubernetes.io/instance:prometheus-operator app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:grafana app.kubernetes.io/version:7.4.5 helm.sh/chart:grafana-6.6.3] map[kubernetes.io/ingress.class:nginx kubernetes.io/tls-acme:true meta.helm.sh/release-name:prometheus-operator meta.helm.sh/release-namespace:monitoring] [] []  []},Spec:IngressSpec{Backend:nil,TLS:[]IngressTLS{IngressTLS <cut>

Any suggestions?

Better error / documenting error

I am using the mutation webhook and currently returning an error like so:

fmt.Fprintf(os.Stderr, "error serving webhook: %s", err)

however, the pod event looks like this:

24m         Warning   FailedCreate        replicaset/hello-secrets-78445d48c9   Error creating: Internal error occurred: failed calling webhook "pods.vault-secrets-webhook.admission.server.com": an error on the server ("{\"response\":{\"uid\":\"97705fda-61c0-11e9-ad59-0800272e4a1f\",\"allowed\":false,\"status\":{\"metadata\":{},\"status\":\"Failure\",\"message\":\"Error getting vault address - make sure you set the annotation\"}}}") has prevented the request from succeeding

is there any way we can only show the error.status.message instead of this ugly json?

How to get Verb/UserInfo/.... when implementing a validate webhook

the validator interface containsValidate(ctx context.Context, obj metav1.Object)
it means i can implement my logic with the resource, nothing else

if i have some logic for the verb, i can't determine what's the verb of this request, i can't get it.....

i think there should be a way for me to get the AdmissionReview struct, but i can't find any examples

Mutating webhook should support setting `Allowed: false`

Currently, slok/kubewebhook only supports StopChain and MutatedObject, or returning 500 ISE directly, but it does not support rejecting the request directly in the validating webhook style.

type MutatorResult struct {
// StopChain will stop the chain of validators in case there is a chain set.
StopChain bool
// MutatedObject is the object that has been mutated. If is nil, it will be used the one
// received by the Mutator.
MutatedObject metav1.Object
// Warnings are special messages that can be set to warn the user (e.g deprecation messages, almost invalid resources...).
Warnings []string
}

I have confirmed that returning {response: {allowed: false}} in mutating webhooks has a similar effect as validating webhooks.

As explained in #188, kube-apiserver handles the response differently if webhook server responds with 200 or with 500. In particular, in my use case, I am adding alerts for the apiserver_admission_webhook_rejection_count metric with error_type=calling_webhook_error, which cannot distinguish upstream-side 500 errors from proxy-side 502 errors. Allowing returning allowed: false in the mutating webhook allows me to collect the correct metrics.

Adding

Currently Mutator is defined as

/ Mutator knows how to mutate the received kubernetes object.
type Mutator interface {
        // Mutate will received a pointr to an object that can be mutated
        // mutators are grouped in chains so the mutate method can return
        // a stop boolean to stop executing the chain and also an error.
        Mutate(context.Context, metav1.Object) (stop bool, err error)
}

This means that some information about the request is lost. It would be good if the actual AdmissionReview itself was made available. This would enable the mutator to look at some fields such as Namespace that is not available in the object itself.

Would you accept a PR that provides this information?

Example in the Create request show below, the namespace is not available in the pod definition.

{
  "kind": "AdmissionReview",
  "apiVersion": "admission.k8s.io/v1beta1",
  "request": {
    "uid": "1a41e160-f369-11e8-b020-000d3afdac62",
    "kind": {
      "group": "",
      "version": "v1",
      "kind": "Pod"
    },
    "resource": {
      "group": "",
      "version": "v1",
      "resource": "pods"
    },
    "namespace": "default",
    "operation": "CREATE",
    "userInfo": {
      "username": "system:serviceaccount:kube-system:replicaset-controller",
      "uid": "a0e767e4-f368-11e8-b020-000d3afdac62",
      "groups": [
        "system:serviceaccounts",
        "system:serviceaccounts:kube-system",
        "system:authenticated"
      ]
    },
    "object": {
      "metadata": {
        "generateName": "php-apache-runc-878945998-",
        "creationTimestamp": null,
        "labels": {
          "pod-template-hash": "878945998",
          "run": "php-apache-runc"
        },
        "ownerReferences": [
          {
            "apiVersion": "apps/v1",
            "kind": "ReplicaSet",
            "name": "php-apache-runc-878945998",
            "uid": "1a3ab45e-f369-11e8-b020-000d3afdac62",
            "controller": true,
            "blockOwnerDeletion": true
          }
        ]
      },
      "spec": {
        "volumes": [
          {
            "name": "default-token-lnbmt",
            "secret": {
              "secretName": "default-token-lnbmt"
            }
          }
        ],
        "containers": [
          {
            "name": "php-apache",
            "image": "k8s.gcr.io/hpa-example",
            "ports": [
              {
                "containerPort": 80,
                "protocol": "TCP"
              }
            ],
            "resources": {
              "requests": {
                "cpu": "200m"
              }
            },
            "volumeMounts": [
              {
                "name": "default-token-lnbmt",
                "readOnly": true,
                "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "Always"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "serviceAccountName": "default",
        "serviceAccount": "default",
        "securityContext": {
          
        },
        "schedulerName": "default-scheduler",
        "tolerations": [
          {
            "key": "node.kubernetes.io/not-ready",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          },
          {
            "key": "node.kubernetes.io/unreachable",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          }
        ],
        "priority": 0
      },
      "status": {
        
      }
    },
    "oldObject": null,
    "dryRun": false
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.