Comments (10)
Kaezon
commented on May 29, 2024
1
@chilu49 @tobymilne-haven
I created a temporary work-around in a branch: Kaezon/k-rail@9599c67
All I did was limit the webhook to processing DELETEs to CRDs. This was because it's the only thing I'm aware of that has a plugin which looks at deletes.
In the long run, I would not keep this solution in place since we probably want k-rail to be examining all requests anyways.
I'm going to see if I can figure out what the actual cause of the problem is and fix it.
from k-rail.
Kaezon
commented on May 29, 2024
I just ran into the same issue trying to delete a Job pod
Kubernetes version: 1.19
k-rail version: v3.5.1
from k-rail.
chilu49
commented on May 29, 2024
I installed k-rail using helm.
So, i ended up doing helm uninstall k-rail after which I was able to delete the pod.
Not sure if this works for you or not.
from k-rail.
chilu49
commented on May 29, 2024
I just ran into the same issue trying to delete a Job pod
Kubernetes version: 1.19
k-rail version: v3.5.1
I installed k-rail using helm.
So, i ended up doing helm uninstall k-rail after which I was able to delete the pod.
Not sure if this works for you or not.
from k-rail.
Kaezon
commented on May 29, 2024
I installed k-rail using helm.
So, i ended up doing helm uninstall k-rail after which I was able to delete the pod.
Not sure if this works for you or not.
Oh, yes. I can remove k-rail to delete the pod; however, deleting k-rail every time I run a job doesn't seem like an ideal way to administrate my deployments :P
from k-rail.
tobymilne-haven
commented on May 29, 2024
I had the same issue as soon as I switched to reportonly false, in the end i hacked the helm chart, and disabled the webhook for "DELETE", that allows pods to be deleted, but i suspect rules about eviction etc wont work.
from k-rail.
Kaezon
commented on May 29, 2024
After adding a lot of debug prints, I found what's happening at least.
It looks like k-rail is trying to attach some extra metadata to the DELETE request. Specifically "seccomp.security.alpha.kubernetes.io/pod:runtime/default"
I'm still not sure why though.
{"kind":"AdmissionReview","apiVersion":"admission.k8s.io/v1","request":{"uid":"f48f2da7-6e29-4d50-bd41-5843bd91a045","kind":{"group":"","version":"v1","kind":"Pod"},"resource":{"group":"","version":"v1","resource":"pods"},"requestKind":{"group":"","version":"v1","kind":"Pod"},"requestResource":{"group":"","version":"v1","resource":"pods"},"name":"banana-app-c74b498db-cps64","namespace":"default","operation":"DELETE","userInfo":{"username":"system:serviceaccount:argocd:argocd-server","uid":"3c627c97-ddae-4f57-baa4-3937d7abcdf4","groups":["system:serviceaccounts","system:serviceaccounts:argocd","system:authenticated"]},"object":null,"oldObject":{"kind":"Pod","apiVersion":"v1","metadata":{"name":"banana-app-c74b498db-cps64","generateName":"banana-app-c74b498db-","namespace":"default","uid":"1a4921ba-456a-48ef-9e25-33a18177222a","resourceVersion":"76855","creationTimestamp":"2021-11-02T18:04:42Z","labels":{"app":"banana","pod-template-hash":"c74b498db"},"annotations":{"cni.projectcalico.org/podIP":"10.1.9.216/32","cni.projectcalico.org/podIPs":"10.1.9.216/32","sidecar.istio.io/inject":"true"},"ownerReferences":[{"apiVersion":"apps/v1","kind":"ReplicaSet","name":"banana-app-c74b498db","uid":"c1aa8378-137e-4c9a-a948-256236e889e4","controller":true,"blockOwnerDeletion":true}],"managedFields":[{"manager":"kube-controller-manager","operation":"Update","apiVersion":"v1","time":"2021-11-02T18:04:42Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:sidecar.istio.io/inject":{}},"f:generateName":{},"f:labels":{".":{},"f:app":{},"f:pod-template-hash":{}},"f:ownerReferences":{".":{},"k:{\"uid\":\"c1aa8378-137e-4c9a-a948-256236e889e4\"}":{".":{},"f:apiVersion":{},"f:blockOwnerDeletion":{},"f:controller":{},"f:kind":{},"f:name":{},"f:uid":{}}}},"f:spec":{"f:containers":{"k:{\"name\":\"banana-app\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:ports":{".":{},"k:{\"containerPort\":8080,\"protocol\":\"TCP\"}":{".":{},"f:containerPort":{},"f:protocol":{}}},"f:resources":{".":{},"f:limits":{".":{},"f:cpu":{},"f:memory":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{}}},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsNonRoot":{},"f:runAsUser":{}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsNonRoot":{},"f:runAsUser":{}},"f:terminationGracePeriodSeconds":{}}}},{"manager":"calico","operation":"Update","apiVersion":"v1","time":"2021-11-02T18:04:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:cni.projectcalico.org/podIP":{},"f:cni.projectcalico.org/podIPs":{}}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2021-11-02T18:04:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.1.9.216\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"default-token-l4qr4","secret":{"secretName":"default-token-l4qr4","defaultMode":420}}],"containers":[{"name":"banana-app","image":"packages.bco.cudaops.com:443/docker-virtual/hashicorp/http-echo@sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96","args":["-listen=:8080","-text=banana"],"ports":[{"containerPort":8080,"protocol":"TCP"}],"resources":{"limits":{"cpu":"550m","memory":"2560Mi"},"requests":{"cpu":"500m","memory":"2Gi"}},"volumeMounts":[{"name":"default-token-l4qr4","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"runAsUser":1000,"runAsGroup":1000,"runAsNonRoot":true}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"bcostabile-barracuda","securityContext":{"runAsUser":1000,"runAsGroup":1000,"runAsNonRoot":true},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-11-02T18:04:42Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-11-02T18:04:43Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-11-02T18:04:43Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2021-11-02T18:04:42Z"}],"hostIP":"192.168.1.189","podIP":"10.1.9.216","podIPs":[{"ip":"10.1.9.216"}],"startTime":"2021-11-02T18:04:42Z","containerStatuses":[{"name":"banana-app","state":{"running":{"startedAt":"2021-11-02T18:04:43Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"sha256:a6838e9a6ff6ab3624720a7bd36152dda540ce3987714398003e14780e61478a","imageID":"packages.bco.cudaops.com:443/docker-virtual/hashicorp/http-echo@sha256:ba27d460cd1f22a1a4331bdf74f4fccbc025552357e8a3249c40ae216275de96","containerID":"containerd://a756072973f9ba3a3d4d7b5222aaf53e1cef82ada12a5e4d82d0fa8575b7f183","started":true}],"qosClass":"Burstable"}},"dryRun":false,"options":{"kind":"DeleteOptions","apiVersion":"meta.k8s.io/v1","gracePeriodSeconds":30,"propagationPolicy":"Foreground"}}}
DEBUG: Printing list of patches
{add /metadata/annotations map[seccomp.security.alpha.kubernetes.io/pod:runtime/default]}
from k-rail.
Kaezon
commented on May 29, 2024
Ok, a little more debugging revealed it's the pod_default_seccomp_policy plugin.
I'll look at the code there next.
DEBUG: List of patches from pod_default_seccomp_policy policy
{add /metadata/annotations map[seccomp.security.alpha.kubernetes.io/pod:runtime/default]}
from k-rail.
funkypenguin
commented on May 29, 2024
I've found this problem as well, after enabling the pod_default_seccomp_policy. The pods were already running, and so thereafter any attempts to delete them caused the above-mentioned issue.
from k-rail.
mark-adams
commented on May 29, 2024
👋 The k-rail project has been deprecated and is no longer under active development. We recommend taking a look at OPA Gatekeeper to see if it might meet your needs going forward.
Thanks for your contribution(s) to the project!
from k-rail.
Related Issues (20)
- Question: how to change the values.yaml file?
- helm install --debug k-rail k-rail/k-rail --namespace k-rail fails HOT 2
- Exemptions do not cover DaemonSets HOT 1
- "runAsNonRoot: true" should be in Pod and Container SecurityContexts HOT 5
- exemptions on container level HOT 2
- Helm Install: no matches for kind "PodDisruptionBudget" in version "policy/v1beta1" HOT 1
- [FR] Emergency stop button. Prevent all changes when toggled. HOT 3
- exempt_policies pod_no_exec and execute to pod/container fails HOT 5
- Can we use regexs in exemptions? HOT 2
- [FR] Make terminationMessagePolicy: FallbackToLogsOnError default HOT 1
- EmptyDir sizelimits no longer applied via mutation HOT 2
- Include violating image in logs produced by pod_trusted_repository policy HOT 2
- Bug in Namespace Process Sharing HOT 1
- Pod policy check inconsistencies HOT 3
- [FR] Add policy to enforce unique Istio VirtualServices (like unique Ingress policy) HOT 3
- [Question] Is there a way to enforce only matching resources on a set of nodes? HOT 3
- Update deprecated api group admissionregistration.k8s.io/v1beta1 HOT 1
- k-rail Webhook Fails Due to TLS SANs Issue + Temporary Workaround HOT 1
- Exemptions and Fuzzy Matching - Possible Bug HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k-rail.