The aim of this work is using Advanced Cluster Management for Kubernetes (ACM) to automate the installation of the CNF operators and their specific configuration in the target clusters.

The process is basically summed up in:

• The installation of CNF operators is done using ACM policies.
• The configuration of CNF operators is done using ACM GitOps approach where the configuration is applied from a Git repository. Therefore, we will make use of Channels, Subscriptions, Applications and PlacementRules objects to tell ACM to configure our CNF components.

• ACM installed in a OpenShift cluster. In my tests we run the following versions:

$ oc get nodes -o wide
NAME       STATUS   ROLES            AGE   VERSION           INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                     CONTAINER-RUNTIME
eko6       Ready    worker           8d    v1.19.0+db1fc96   10.19.139.35   <none>        Red Hat Enterprise Linux CoreOS 46.82.202010022240-0 (Ootpa)   4.18.0-193.24.1.el8_2.dt1.x86_64   cri-o://1.19.0-20.rhaos4.6.git97d715e.el8
eko7       Ready    worker           9d    v1.19.0+db1fc96   10.19.139.36   <none>        Red Hat Enterprise Linux CoreOS 46.82.202010022240-0 (Ootpa)   4.18.0-193.24.1.el8_2.dt1.x86_64   cri-o://1.19.0-20.rhaos4.6.git97d715e.el8
master-0   Ready    master,virtual   9d    v1.19.0+db1fc96   10.19.140.20   <none>        Red Hat Enterprise Linux CoreOS 46.82.202010022240-0 (Ootpa)   4.18.0-193.24.1.el8_2.dt1.x86_64   cri-o://1.19.0-20.rhaos4.6.git97d715e.el8
master-1   Ready    master,virtual   9d    v1.19.0+db1fc96   10.19.140.21   <none>        Red Hat Enterprise Linux CoreOS 46.82.202010022240-0 (Ootpa)   4.18.0-193.24.1.el8_2.dt1.x86_64   cri-o://1.19.0-20.rhaos4.6.git97d715e.el8
master-2   Ready    master,virtual   9d    v1.19.0+db1fc96   10.19.140.22   <none>        Red Hat Enterprise Linux CoreOS 46.82.202010022240-0 (Ootpa)   4.18.0-193.24.1.el8_2.dt1.x86_64   cri-o://1.19.0-20.rhaos4.6.git97d715e.el8

• As target or remote clusters I imported an OpenShift cluster installed with the same nightly build

❗ Importing a cluster can be done using the ACM web user interface or using the CLI in a more scripted way. Take a look to the following notes on how to import clusters from command line.

$ oc get nodes
NAME                                              STATUS   ROLES               AGE     VERSION
cnf10-master-0.cnf10.kni.lab.eng.bos.redhat.com   Ready    master,virtual      9d      v1.19.0+db1fc96
cnf10-master-1.cnf10.kni.lab.eng.bos.redhat.com   Ready    master,virtual      9d      v1.19.0+db1fc96
cnf10-master-2.cnf10.kni.lab.eng.bos.redhat.com   Ready    master,virtual      9d      v1.19.0+db1fc96
cnf10-worker-0.dev5.kni.lab.eng.bos.redhat.com    Ready    worker,worker-cnf   6d17h   v1.19.0+db1fc96
cnf11-worker-0.dev5.kni.lab.eng.bos.redhat.com    Ready    worker,worker-cnf   9d      v1.19.0+db1fc96

⚠️ The imported cluster must have worker nodes SR-IOV and PTP capables otherwise the operators deployed won't actually configure the hardware appropiately. If they are not, you still can install the different operators but you won't be able to configure them.

In our scenario we have an ACM hub cluster installed and a spoke cluster running OpenShift already imported in ACM. So, ACM is ready to manage the spoke cluster.

Before digging into how to install and configure each of the CNF operators we need to define the structure of this repository. Following GitOps best practices it is divided into different branches:

❗ The branch you are on now is master :)

• Master branch. In this branch it is stored all the ACM specific manifests. This means: policies, placement rules, applications... ,i.e. all the resources that ACM understand-
• Stage branch. Here, we can find all the Kubernetes native objects or specific CNF custom resources required to configure the different operators. This includes for instance the ptpconfig files, performance profiles for the Performance Addon Operator, SRIOV network definitions, etc. Another key point is that they are stored in a branch called stage, which means that this is going to be the configuration for each CNF operator that is going to be applied to all staging OpenShift clusters.

⚠️ See that you can create new branches for clusters in development, production or even per hardware vendor or model. It is up to you how you group your different clusters. The only thing you need to take into account is that clusters grouped will share the same CNF configuration.

• Config branch. This branch is used to store a baseline CNF configuration for each operator. Then, using kustomize we can replace or change parts of this configuration in order to set exactly the right configuration for each environment. For instance, we can have a basic PTP configuration in the config branch and when applying it in staging clusters some parts of this configuration are replaced.

❗ Not all CNF operators require a default or base configuration. SCTP for instance is using a base configuration stored in the config branch, which applies a replacement depending on the environment where the cluster is grouped.

Here are the different CNF components that are installed, configured and managed by ACM. Click on the one you are interested on to see the detailed configuration process:

• Performance Addon Operator
• PTP Operator
• SRIOV Network Operator
• DPDK
• SCTP module