Cilium installation with BGP Control Plane enabled on OpenShift fails about cilium HOT 8 CLOSED

I just ran into the same issue when trying to enable the BGP control plane on an existing OCP cluster. Applying the workaround as mentioned in #31499 (comment) worked for me as well.

I had a quick look at the role that's deployed for the OLM operator service account (that role grants the OLM SA verbs: ['*'] which is a superset of the permissions of the denied role). However, it seems like something in Kubernetes doesn't realize that creating RBAC with specific verbs for a resource isn't a privilege escalation for a principal which holds permissions for verbs: ['*'] for that resource.

Edit: I didn't read the comments carefully enough earlier. The actual problem is that the current OLM RBAC only works if Cilium is installed in namespace kube-system (at least for Cilium 1.14), since the Helm chart used by the OLM install for 1.14 doesn't support customizing the bgp-secrets-namespace flag which defaults to kube-system. If the OLM operator runs in a different namespace (e.g. cilium for us), it doesn't have sufficient permissions to create a Role or RoleBinding to access secrets in namespace kube-system out of the box.

I haven't checked the Helm chart for 1.15.1 yet, but from the docs it seems like with 1.15 this issue can be avoided by setting Helm value bgpControlPlane.secretNamespace.name to the name of the namespace in which Cilium is installed.

from cilium.

I also tried an approach to install OCP with Cilium CNI without bgp control plane enabled That works fine and cluster is up with Cilium CNI. However, when ciliumconfig CR is changed with bgpControlPlaned enabled, then OLM claims that cluster upgrade has failed with the crash

Did you check the cilium agent logs to see what the issue might be? Please attach a sysdump.

from cilium.

I think it might be related to missing clusterrole and role binding

I solved it by adding the following

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cilium-cilium-olm-secrets
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cilium-cilium-olm-secrets
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cilium-cilium-olm-secrets
subjects:
- kind: ServiceAccount
  name: cilium-olm
  namespace: cilium

then after restart of the cilium agents, config is reconciled, new images pulled and bgp is ready to be used

[root@cilium-installer ~]# oc create -f bgp-policy.yaml
ciliumbgppeeringpolicy.cilium.io/xrd created
[root@cilium-installer ~]# cilium bgp peers --namespace cilium
Node                          Local AS   Peer AS   Peer Address   Session State   Uptime   Family         Received   Advertised
cilium-2vx97-master-0         64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0
cilium-2vx97-master-1         64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0
cilium-2vx97-master-2         64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0
cilium-2vx97-worker-0-9cnpm   64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0
cilium-2vx97-worker-0-gf8jj   64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0
cilium-2vx97-worker-0-p4dff   64513      64512     192.168.66.1   active          0s       ipv4/unicast   0          0
                                                                                           ipv6/unicast   0          0

from cilium.

That's odd, we should be configuring those values correctly.

from cilium.

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

from cilium.

This issue has not seen any activity since it was marked stale.
Closing.

from cilium.

Issue has been reported here
isovalent/olm-for-cilium#91

from cilium.

Using these values, should mean that you are not getting the warning message, even if you don't intend to use BGP secrets.

    bgpControlPlane:
      enabled: true
      secretsNamespace:
        name: cilium
        create: false

from cilium.