Hi folks,

This is another change that we think might be good to be included in the official repo. The point here is that before applying PAO or SR-IOV or any other operator configuration we want to check that the operator is actually deployed.

In order to do that we included a preSync hook in each operator config folder. An example of PAO configuration (profile + presync) can be found here. See that first we are just checking that the operator is Running and when it is running the Sync phase can start, basically applying the performance profile. The same has been configured for SRIOV here

In order to do that we need to configure a couple of things in the remote cluster so we can execute "oc commands" locally. Basically, we need to apply the clusterRole and clusterRolebindings that you already created in the management cluster. We included these permissions in the base configuration of the remote clusters, see a working example here . In that case, we are just calling the same ClusterRoles and ClusterRoleBindings used for the management cluster (notice that oc client must be in 4.8.x version otherwise HTTPS files won't be applied). Another approach is just to leave both files in the base remote cluster folder as shown here

So the point is that these permissions will allow us to run pre and post-sync hooks in remote clusters as well. Also, it will introduce some logic when it comes to applying configuration since it won't apply if the operator is still installing. I think this should be included with all the operators that require post configuration.

Let me know your thoughts and how I can contribute.

In non-dense clusters (like Core and non-DU clusters), it is possible to use an in-cluster ArgoCD instance to do pull of configurations. This model removes dependencies on a centralized instance and allow for highly distributable and scalable GitOps model.

The main caveats that need to be addressed for this model are:

• the unknown resource utilization brackets to make this work on a cluster based on number of nodes, MCPs and CNFs
• the lack of centralized visibility to be aware of the status of each clusters. This could be address by RHACM but need to be documented and tested?
• how to maintain hierarchical GitOps model between organization desired configurations vs localized customizations?
• when can a centralized deployment GitOps be used in conjunction to a distributed in-cluster config & operations GitOps?

There is are design considerations (like labeling scheme, mono-repo vs multi-repo, the type and privileges of ArgoCD instances, etc), that are built into the design of the repo but no public information on them is available. This RFE is for:

• How to use telco-gitops repo on various configurations for RAN & Core deployments (Combined, CU pool, DU pool, SNO DU, Core, etc)
• Include documentation of labeling scheme and how to extend it
• Include documentation ArgoCD instances and privileges
• Include documentation of service accounts and roles
• Include documentation on options for mgmt clusters

Use public reference from clusters by @williamcaban and @novacain1

It appears that any patch files (look under telco-gitops/base/operators/openshift-sriov-network-operator/kustomization.yaml) or (telco-gitops/blueprints/combined-ran/kustomization.yaml) break rendering under OpenShift clients based on 4.8+.

oc kustomize .

Error: accumulating resources: accumulation err='accumulating resources from 'github.com/novacain1/telco-gitops/blueprints/combined-ran?ref=main': evalsymlink failure on '/home/dcain/carslab-public/rhocp-clusters/skylark.cars.lab/github.com/novacain1/telco-gitops/blueprints/combined-ran?ref=main' : lstat /home/dcain/carslab-public/rhocp-clusters/skylark.cars.lab/github.com: no such file or directory': recursed accumulation of path '/tmp/kustomize-435906337/blueprints/combined-ran': no matches for IdId operators.coreos.com_v1alpha1_Subscription|~X|sriov-network-operator-subscription; failed to find unique target for patch operators.coreos.com_v1alpha1_Subscription|sriov-network-operator-subscription

Removing patch files from the above two sources in telco-gitops restores rendering.

Believe there was a major change in the clients from k8s 1.20 (ocp 4.7) to now:
https://github.com/kubernetes-sigs/kustomize#kubectl-integration

The Special Resource Operator (SRO) installs NFD. Need to validate behavior does not present conflicts when installing SRO in a clusters where NFD Operator is already running.

FYI @novacain1 @williamcaban

The labelling approach with Kustomize assumes the resource definition is part of the bases or manifests and for that reason does not work well for existing elements like nodes, Pods or deployments.

The labelling flow should account for:

• Follow GitOps principles and have a continuous remediation cycle
• Work for labeling any objects that already exists on a cluster (e.g. nodes, secrets, pods, etc)
• Should work when the consumed by GitOps controllers

Example of options to investigate:

• The use of Kustomize Transformers
• Using OPA Gatekeeper (https://github.com/open-policy-agent/gatekeeper)
• Using resource-locker-operator (https://github.com/redhat-cop/resource-locker-operator)

Hi,

We have been using this custom resource that has been included recently in RHACM that allows provisioned clusters to be managed by the central ArgoCD instance installed in the hub cluster.

Here is a working example of the GitOpsCluster CR that needs to be applied to the management cluster. Once added you need to include the following label into the ManagedCluster CR for every cluster that wants to be managed by the central Argo instance.

I think that with this new CR you can skip some manual steps included in the README that allows to automatically include the recently installed cluster into central ArgoCD. I am not sure where it should be added to your repo since RHACM installation is a requirement. In my forked repository, it is included in the management repo since it is something that is going to be installed in the management cluster, actually inside the config-ztp folder.

Let me know your thoughts.