cloud-ingress-operator is designed to assist in toggling OpenShift Dedicated 4.x clusters "private" and "public" through the use of custom Kubernetes resources.
There are two pieces of the cluster which can be toggled in this way: The default API server (api.<cluster-domain>
) and the ingresses (the default being *.apps.<cluster-domain>
, and up to one additional, named *.apps2.<cluster-domain>
). These may be handled independently of one another in the manner described below.
To ensure that Hive and SRE have continued access to manage the cluster on behalf of the customer, an additional API endpoint is created, named by default rh-api.<cluster-domain>
. This endpoint may also be referred to as the "admin endpoint," or "admin API endpoint."
The operator does not manage any kind of VPC peering or VPN connectivity into the clustered environment.
As mentioned above, the operator is controlled through custom Kubernetes resources, APIScheme
and PublishingStrategy
. They are documented below:
The APIScheme resource instructs the operator to create the admin API endpoint. The specification of the resource is explained below:
spec:
managementAPIServerIngress:
enabled: true
dnsName: rh-api
allowedCIDRBlocks:
- "0.0.0.0/0"
In this example, the endpoint will be called rh-api
and the full name rh-api.<cluster-domain>
. Furthermore, there will be a single entry in the security group associated with the cloud load balancer that allows 0.0.0.0/0
(everything).
Toggling privacy is done with the PublishingStrategy
custom resource.
spec:
defaultAPIServerIngress:
listening: external
applicationIngress:
- listening: external
default: true
dnsName: "*.apps"
certificate:
secretRef:
name: foo
routeSelector:
labelSelector:
matchLabels:
foo: bar
In this example, the default API server (api.<cluster-domain>
) is configured to be externally available ("public") and the default ingress (named *.apps.<cluster-domain>
, and identified by default: true
) is configured to also be externally available ("public"). Additionally, the default ingress is configured to use a TLS certificate named foo
, in the openshift-ingress
namespace. Finally, the ingress uses the route selector as specified.
Note: the namespace
attribute for secretRef
is not currently used; certificates must be within the openshift-ingress
namespace.
It is possible to add additional applicationIngresses, however at this time, OSD supports the default plus an additional.
Due to a race condition with the cluster-ingress-operator we test the logic flow of ingresscontroller manually. Once you are in a cluster, here are the steps to do so:
- Pause syncset to the cluster SOP
- Check the inital state of the ingresscontrollers on cluster before the test by running
oc get ingresscontrollers -n openshift-ingress-operator
- In this test, we assume there is only one ingresscontroller called
default
.
- In this test, we assume there is only one ingresscontroller called
- Apply a sample
PublishingStrategy
CR with these specs
spec:
defaultAPIServerIngress:
listening: external
applicationIngress:
- listening: external
default: true
dnsName: "apps.sample.default.domain"
certificate:
secretRef:
name: foo
- listening: internal
default: false
dnsName: "apps2.sample.nondefault.domain"
certificate:
secretRef:
name: bar
routeSelector:
labelSelector:
matchLabels:
foo: bar
- Looking at
applicationIngress
, the expected result will be the creation of 2 ingresscontrollers,default
andapps2
. Thedefault
ingresscontroller will have all the attributes of the firstapplicationIngress
replacing the olddefault
ingresscontroller, andapps2
will have the attributes of the secondapplicationIngress
. To check these results, runoc get ingresscontrollers -n openshift-ingress-operator
and view eachingresscontroller
as yaml. - NOTE: it might take up to 60 seconds for these changes to apply due to a race condition.
The operator is built with operator-sdk. There is a tight dependency on the AWS cluster provider but the dependency is pinned to the OpenShift fork for access to v1beta1 API features.
You can quickly debug the operator on your existing OSD cluster by following the below steps. It is recommended to do this against a staging cluster.
-
Connect to your cluster through backplane or directly
-
Elevate your permissions:
oc adm groups add-users osd-sre-cluster-admins $(oc whoami)
-
Scale down
cluster-version-operator
andcloud-ingress-operator
oc scale --replicas 0 -n openshift-cluster-version deployments/cluster-version-operator
oc scale --replicas 0 -n openshift-cloud-ingress-operator deployments cloud-ingress-operator
- Debug! If you are using VSCode, create/update your
launch.json
as followed
{
"version": "0.2.0",
"configurations": [{
"type": "go",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/cmd/manager/main.go",
"env":{
"WATCH_NAMESPACE": "openshift-sre-sshd,openshift-cloud-ingress-operator,openshift-ingress,openshift-ingress-operator,openshift-kube-apiserver,openshift-machine-api"
}
}]
}