The openshift-apiserver operator installs and maintains the openshift-apiserver on a cluster

License: Apache License 2.0

Dockerfile 0.42% Makefile 2.18% Go 97.41%

cluster-openshift-apiserver-operator's Introduction

OpenShift API Server Operator

The OpenShift API Server operator manages and updates the OpenShift API Server. The operator is based on OpenShift library-go framework and it is installed via Cluster Version Operator (CVO).

It contains the following sub-systems:

Operator
Configuration observer

By default, the operator exposes Prometheus metrics via metrics service. The metrics are collected from following components:

OpenShift API Server Operator

Configuration

The configuration observer component is responsible for reacting on external configuration changes. For example, this allows external components (registry, etc..) to interact with the OpenShift API server configuration (OpenShiftAPIServerConfig custom resource).

Currently changes in following external components are being observed:

cluster images.config.openshift.io custom resource
- The observed CR resource is used to configure the imagePolicyConfig.internalRegistryHostname in Kubernetes API server configuration
cluster projects.config.openshift.io custom resource
- The observed CR resource is used to configure the Project request defaults
cluster ingress.config.openshift.io custom resource
- The observed CR resource is used to set routingConfig.subdomain in the OpenShift API server configuration.

The configuration for the OpenShift API server is the result of merging:

a default config
observed config (compare observed values above) spec.spec.unsupportedConfigOverrides from the openshiftapiserveroperatorconfig.

All of these are sparse configurations, i.e. unvalidated json snippets which are merged in order to form a valid configuration at the end.

Debugging

To gather all information necessary for debugging operator please use the must-gather tool.

cluster-openshift-apiserver-operator's People

Contributors

Stargazers

Watchers

Forkers

deads2k soltysh tnozicka juanvallejo mfojtik damemi smarterclayton sanchezl bparees sttts miciah derekwaynecarr sjenning abhinavdahiya jhadvig jboyd01 twiest mrogers950 ravisantoshgudimetla p0lyn0mial enj openshift-cherrypick-robot s-urbaniak stlaz sallyom paulfantom csrwng marun gabemontero haircommander umohnani8 sohankunkerkar jhrozek jmrodri jaormx openshift-kni danwinship vareti kodieglosseribm tkashem lilic ironcladlou retroflexer openshift-bot abhat vpickard dustman9000 rcarrillocruz dmage jupierce guillaumerose cfergeau osherdp global-localhost global19 global19-atlassian-net emilym1 lack huffmanca isabella232 ingvagabund alainlompo vrutkovs dgrisonnet atiratree eggfoobar isgasho lm0943111262 qjkee xueqzhan aojea dinhxuanvu ibihim rhmdnd cdoern thejasn danielkucera marioferh lance5890 simonpasquier liouk openshift-art-build-bot prashanth684 knelasevero

cluster-openshift-apiserver-operator's Issues

operator does not upgrade apiserver daemonset

After an upgrade, the operator is not updating the apiserver daemonset

$ oc get clusterversion
NAME      VERSION                           AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.0.0-0.alpha-2019-02-08-113402   True        False         37m     Cluster version is 4.0.0-0.alpha-2019-02-08-113402

$ oc get pod -oyaml openshift-apiserver-operator-596b99d569-6hbgw 
...
spec:
  containers:
...
    env:
    - name: IMAGE
      value: registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-08-113402@sha256:a3e3f16ed997d16de00d99e9181fcab676a7cdd0f1b8bd99b432bf0092754b6a
    image: registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-08-113402@sha256:28d21c510802e4e4733e095a01ee22c1fe58ca76ce3f6515c7157fc19ead645f

$ oc get ds -oyaml apiserver 
spec:
  template:
    spec:
      containers:
        image: registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-08-055616@sha256:a3e3f16ed997d16de00d99e9181fcab676a7cdd0f1b8bd99b432bf0092754b6a  <--- doesn't match cluster version

@derekwaynecarr @smarterclayton @deads2k

Move TestingEventRecorder to a subpackage under spot where the in memory one is

Originally posted by @deads2k in #94

update imageregister defaultRoute will make openshift api restart.

See the summary above, after enable/disable the imageregistry defaultRoute, the openshift apiserver will be restart.

[root@ocp420219wtj-inf ~]# oc patch configs.imageregistry.operator.openshift.io/cluster --type merge -p '{"spec":{"defaultRoute":false}}'
config.imageregistry.operator.openshift.io/cluster patched
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
NAME              READY   STATUS    RESTARTS   AGE
apiserver-rgcfb   1/1     Running   0          46h
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
NAME              READY   STATUS        RESTARTS   AGE
apiserver-rgcfb   1/1     Terminating   0          46h
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
NAME              READY   STATUS    RESTARTS   AGE
apiserver-bpd2l   0/1     Pending   0          1s
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
NAME              READY   STATUS     RESTARTS   AGE
apiserver-bpd2l   0/1     Init:0/1   0          2s
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
error: You must be logged in to the server (Unauthorized)
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
error: You must be logged in to the server (Unauthorized)
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
error: You must be logged in to the server (Unauthorized)
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
error: You must be logged in to the server (Unauthorized)
[root@ocp420219wtj-inf ~]# oc get pods -n openshift-apiserver
error: You must be logged in to the server (Unauthorized)

Access to a privileged container allows for breakout to the underlying host

1 as the openshift-apiserver has set the 'privileged: true'

cluster-openshift-apiserver-operator/bindata/v3.11.0/openshift-apiserver/deploy.yaml

Lines 89 to 93 in 792afb9

 # we need to set this to privileged to be able to write audit to /var/log/openshift-apiserver 

 securityContext: 

 privileged: true 

 readOnlyRootFilesystem: false 

 runAsUser: 0

2 but even without the 'privileged: true', the openshift-apiserver can also write audit to /var/log/openshift-apiserver

3 when using standard container runtimes (for example ContainerD or CRI-O) access to a privileged container allows for easy breakout to the underlying host, which in turn allows for access to all other workloads on that host and credentials for the node agent (Kubelet)

maybe we should remove the "privileged: true"

Future Release Branches Frozen For Merging | branch:release-4.5 branch:release-4.4

The following branches are being fast-forwarded from the current development branch (master) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

release-4.5
release-4.4

Contact the Test Platform or Automated Release teams for more information.

Operator fails with 'Generation: expected: 3, actual: 5'

See https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/openshift_openshift-ansible/10695/pull-ci-openshift-openshift-ansible-master-e2e-gcp/1078

results:
    cmd: /bin/oc get ClusterOperator openshift-cluster-openshift-apiserver-operator -o json -n default
    results:
    - apiVersion: config.openshift.io/v1
      kind: ClusterOperator
      metadata:
        creationTimestamp: '2018-12-06T15:25:26Z'
        generation: 1
        name: openshift-cluster-openshift-apiserver-operator
        resourceVersion: '12310'
        selfLink: /apis/config.openshift.io/v1/clusteroperators/openshift-cluster-openshift-apiserver-operator
        uid: 27de192a-f96b-11e8-928e-42010af00002
      spec: {}
      status:
        conditions:
        - lastTransitionTime: '2018-12-06T15:40:29Z'
          status: 'False'
          type: Failing
        - lastTransitionTime: '2018-12-06T15:40:29Z'
          status: 'True'
          type: Available
        - lastTransitionTime: '2018-12-06T15:40:29Z'
          message: 'Generation: expected: 3, actual: 5'
          reason: DesiredStateNotYetAchieved
          status: 'True'
          type: Progressing
        extension: null
        version: ''

Future Release Branches Frozen For Merging | branch:release-4.17 branch:release-4.18

release-4.17
release-4.18

For more information, see the branching documentation.

Will openshift-apiserver-operator keep openshift-apiserver unchanged

Hi all, I have a question about openshift-apiserver-operator and I would like to ask. Recently I built an openshift cluster. The following is the specific information:

zz:~$ oc get no
NAME       STATUS     ROLES    AGE    VERSION
master-0   Ready      master   5d3h   v1.17.1
master-1   Ready      master   5d3h   v1.17.1
master-2   Ready      master   5d3h   v1.17.1
worker-0   Ready      worker   3d3h   v1.17.1
worker-1   Ready      worker   3d3h   v1.17.1
worker-3   NotReady   worker   3d3h   v1.17.1

zz:~$ oc get po -n openshift-apiserver
NAME                         READY   STATUS    RESTARTS   AGE
apiserver-5564cb8fc6-5zsms   1/1     Running   0          5d2h
apiserver-5564cb8fc6-b5v4g   1/1     Running   0          5d2h
apiserver-5564cb8fc6-cr24p   1/1     Running   0          5h33m
zz:~$ oc get po -n openshift-apiserver
NAME                         READY   STATUS    RESTARTS   AGE
apiserver-5564cb8fc6-5zsms   1/1     Running   0          5d2h
apiserver-5564cb8fc6-b5v4g   1/1     Running   0          5d2h
apiserver-5564cb8fc6-cr24p   1/1     Running   0          5h38m
zz:~$ oc get po -n openshift-apiserver-operator
NAME                                            READY   STATUS    RESTARTS   AGE
openshift-apiserver-operator-68d88d6c75-8x5sv   1/1     Running   0          5d3h

I want to change the run command of the pod of openshift-apiserver, so I tried to modify its corresponding Deployment file. However, although the modification will restart a corresponding pod, but the modified content does not take effect, it is the original command.

I checked the log content of the pod corresponding to openshift-apiserver-operator and knew that openshift-apiserver-operator detected this change.

I0512 01:44:50.069119       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"7db66f01-1b98-4cd0-a0ae-56fe55e09992", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/apiserver -n openshift-apiserver because it changed
I0512 01:44:50.256804       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"7db66f01-1b98-4cd0-a0ae-56fe55e09992", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/openshift-apiserver changed: Progressing changed from False to True ("APIServerDeploymentProgressing: deployment/apiserver.openshift-operator: observed generation is 5, desired generation is 6.")
I0512 01:44:50.593221       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"7db66f01-1b98-4cd0-a0ae-56fe55e09992", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/openshift-apiserver changed: Degraded message changed from "" to "APIServerDeploymentDegraded: 1 of 3 requested instances are unavailable"
I0512 01:45:01.983131       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"7db66f01-1b98-4cd0-a0ae-56fe55e09992", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/openshift-apiserver changed: Progressing changed from True to False ("")
I0512 01:45:12.229744       1 event.go:281] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-apiserver-operator", Name:"openshift-apiserver-operator", UID:"7db66f01-1b98-4cd0-a0ae-56fe55e09992", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/openshift-apiserver changed: Degraded message changed from "APIServerDeploymentDegraded: 1 of 3 requested instances are unavailable" to ""

I want to confirm whether the modified things have been restored due to openshift-apiserver-operator? Thanks!

Need invariant checker for all e2e tests

that ensures that we don't go unavailable. This may be similar to https://github.com/openshift/origin/blob/5e81eb5e7339522258bf1e6b38b6ec4f9fc7e152/test/extended/deployments/deployments.go#L44-L49 if we can find the right spot to plumb it.

Future Release Branches Frozen For Merging | branch:release-4.16 branch:release-4.17

release-4.16
release-4.17

For more information, see the branching documentation.

Return error on RunOperator context done

When the context is done during RunOperator should not the error returned be not null? Is this not an error?

cluster-openshift-apiserver-operator/pkg/operator/starter.go

Line 401 in 00f7e4c

return nil

error synchronizing: clusterrole.rbac.authorization.k8s.io "aggregated-apiserver-clusterrole" not found

@RobertKrawitz is seeing:

Warning Unhealthy 39m kubelet, dev1-master-0 Readiness probe failed: Get https://10.2.0.19:8443/healthz: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Unhealthy 38m kubelet, dev1-master-0 Readiness probe failed: HTTP probe failed with statuscode: 500
E1017 15:21:07.774244 1 cache.go:332] error synchronizing: clusterrole.rbac.authorization.k8s.io "aggregated-apiserver-clusterrole" not found

in the apiserver pod in the openshift-apiserver namespace using openshift-install v0.2.0-23-g079fe51f8f6b4e9ec5f084e620d1adf13c0adbe5 (I dunno what cluster-opernshift-apiserver-operator image) on libvirt. Is this a known issue?

Apiserver startup takes 50+ seconds

Here is a log: https://gist.github.com/sttts/044eeb31338731f9c639d90bf6759980

Every API group take a number of seconds to finish, summing up to nearly a minute, eventually leading the readiness timeouts before #65.

One sees etcd logs for each of them, but it is unclear whether the etcd health check is the reason.

How to configure custom Audit Policy?

Hello,

with Openshift 3.x we had the ability to configure advanced master api audit logging by providing a custom policy:
https://docs.openshift.com/container-platform/3.11/install_config/master_node_configuration.html#master-node-config-advanced-audit
https://kubernetes.io/docs/tasks/debug-application-cluster/audit/

How can this be accomplished with Openshift 4.x ? The apiserver config is supposably generated by the opensshift-apiserver-operator, and provided by the configmap "config" in namespace "openshift-apiserver". But there the default audit policy is provided as JSON inline.

Thanks for your help,
Thomas

	# we need to set this to privileged to be able to write audit to /var/log/openshift-apiserver
	securityContext:
	privileged: true
	readOnlyRootFilesystem: false
	runAsUser: 0

openshift / cluster-openshift-apiserver-operator Goto Github PK