Comments (11)
k8s-ci-robot
commented on September 26, 2024
This issue is currently awaiting triage.
If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.
The triage/accepted label can be added by org members by writing /triage accepted in a comment.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
from kubernetes.
neolit123
commented on September 26, 2024
/sig node
https://github.com/search?q=repo%3Akubernetes%2Fenhancements%20appArmorProfile&type=code
from kubernetes.
haircommander
commented on September 26, 2024
/cc @tallclair
/triage needs-information
I think this is an issue with containerd, your version may not be new enough to take the new field into account. We may need a containerd dev to take a look
from kubernetes.
jbmolle
commented on September 26, 2024
I'm using conatinerd 1.7.17 which is the last stable release from 2 weeks ago. So it would mean that the feature is still experimental in containerd?
from kubernetes.
haircommander
commented on September 26, 2024
yeah I would guess so. @mikebrow do you know?
from kubernetes.
tallclair
commented on September 26, 2024
hmm, the 2 deployments you shared should yield identical pods after the API synchronizes the annotation & field. Can you confirm that the resulting pods from each have both the field & annotations set (to unconfined)?
Also, are your nodes running v1.30?
from kubernetes.
tallclair
commented on September 26, 2024
/cc @tallclair /triage needs-information
I think this is an issue with containerd, your version may not be new enough to take the new field into account. We may need a containerd dev to take a look
Kubelet sets both the new and old version of the apparmor profile in the CRI, so it shouldn't matter
kubernetes/pkg/kubelet/kuberuntime/security_context.go
Lines 44 to 45 in 6911225
from kubernetes.
jbmolle
commented on September 26, 2024
Yes my nodes are running v1.30.1.
I've checked the 2 containers on the node with crictl inspect and the result is not exactly the same.
I've attached the 2 files.
inspection.not-working.json
inspection.working.json
Line 254 in the working container (the one defined with annotations), we have the json fields info.config.linux.security_context.apparmor and info.config.linux.security_context.apparmor_profile. Those fields are not present in the container with appArmorProfile k8s flag.
Line 307 in the not working container we have the field info.runtimeSpec.process.apparmorProfile: cri-containerd.apparmor.d which is not present in the working container
I've actually noticed the problem when I've updated Cilium from v1.15.4 to v1.15.5 because they added the appArmorProfile type Unconfined flag and they removed the annotations. In v1.15.5 my cilium pods were not starting anymore.
from kubernetes.
jbmolle
commented on September 26, 2024
I'm running the 2 following pods:
Pod A
apiVersion: v1
kind: Pod
metadata:
namespace: test
name: test
annotations:
container.apparmor.security.beta.kubernetes.io/busybox: unconfined
spec:
containers:
- name: busybox
image: busybox
command:
- sleep
- "300"
and Pod B
apiVersion: v1
kind: Pod
metadata:
namespace: test
name: test
spec:
containers:
- name: busybox
image: busybox
command:
- sleep
- "300"
securityContext:
appArmorProfile:
type: Unconfined
And then I'm running kubectl apply -f yaml_file_for_pod.yaml -oyaml --dry-run=server
For Pod A which is working, I get the securityContext:
apiVersion: v1
kind: Pod
metadata:
annotations:
container.apparmor.security.beta.kubernetes.io/busybox: unconfined
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{"container.apparmor.security.beta.kubernetes.io/busybox":"unconfined"},"name":"test","namespace":"test"},"spec":{"containers"
:[{"command":["sleep","300"],"image":"busybox","name":"busybox"}]}}
creationTimestamp: "2024-06-10T10:59:43Z"
name: test
namespace: test
uid: 592deaea-11c3-459a-84e1-a0c3bb578708
spec:
containers:
- command:
- sleep
- "300"
image: busybox
imagePullPolicy: Always
name: busybox
resources: {}
securityContext:
appArmorProfile:
type: Unconfined
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-vcl4z
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-vcl4z
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
phase: Pending
qosClass: BestEffort
And for Pod B
apiVersion: v1
kind: Pod
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"test","namespace":"test"},"spec":{"containers":[{"command":["sleep","300"],"image":"busybox","name":"busybox","secu
rityContext":{"appArmorProfile":{"type":"Unconfined"}}}]}}
creationTimestamp: "2024-06-10T11:00:58Z"
name: test
namespace: test
uid: 7e95490b-de34-4242-bc0e-c8c4c272a187
spec:
containers:
- command:
- sleep
- "300"
image: busybox
imagePullPolicy: Always
name: busybox
resources: {}
securityContext: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-vghxr
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-vghxr
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
phase: Pending
qosClass: BestEffort
So the problem must come from func getProfile line 56 in kubernetes/pkg/security/apparmor/helpers.go
I've run the tests from helpers_test.go and they work.
I've also tried to enable explicitly the feature gates AppArmor and AppArmorFields but I've still the same error (which is normal because they are enabled per default anyway).
from kubernetes.
jbmolle
commented on September 26, 2024
With a kind cluster both YAML files give the same result and AppArmor is present in securityContext so it seems to be a problem with my cluster which I've created with kubeadm
from kubernetes.
jbmolle
commented on September 26, 2024
I've started a plain K8s cluster with kubeadm and it works fine.
I've finally looked at the mutating webhooks I had on the cluster and those modifying the pods.
I had Istio and Opentelemetry operator. By removing opentelemetry everything works correctly so I'm going to open an issue on their github.
Thanks @tallclair for your help! Sorry for the false alarm
from kubernetes.
Related Issues (20)
- Prometheus allows "/" in label values but kubernetes does not HOT 5
- [KEP-4191] Support case when writeable layer and readable layers are on same mount but in different locations HOT 6
- kubectl logs -f stop after log rotation HOT 7
- PVC Mounts fails when STAGE_UNSTAGE_VOLUME check fails for k8s EFS CSI HOT 7
- Failed: Build failed outside out of test results HOT 1
- RBAC not work for /healthz HOT 5
- PreStop don't work if hostNetwork set to true HOT 3
- Still seeing the issue for endpoints staying out of sync HOT 16
- Using runtime scheme.Convert for Validating Admission Policy version conversion results in error HOT 4
- When setting kubeReserved non-provided values should fallback to config HOT 3
- option to suppress DNSConfigForming warning log messages and events HOT 2
- CVE PLACEHOLDER ISSUE HOT 3
- Protect from Unauthorized access of Volume HOT 18
- Unable to access control plane after Kubernetes worker node reboot HOT 4
- Failure cluster [03a05e59...] `[sig-network] Networking Granular Checks: Services should function for service endpoints using hostNetwork` HOT 6
- Pod Stuck In Terminating State After Kublet Restart HOT 5
- kubectl >= 1.30.0 triggers leak of goroutines in containerd on `kubectl exec` HOT 38
- Add debug_redact to fields in message AuthConfig (image streaming CRI API) HOT 7
- [FG:InPlacePodVerticalScaling] Change in version-skewed behavior in v1.31 HOT 18
- Memory EmptyDir tmpfs size capped at 100% of node RAM size, not SizeLimit as specified HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kubernetes.