projectcontour / contour-operator Goto Github PK

Please describe the problem you have
Currently, the Contour Deployment example, exposes the debug network port. This is not required and should be exposed conditionally. For example:

...
name: contour
ports:
- containerPort: 8000
  name: debug
  protocol: TCP
...

/cc @stevesloka @jpeach @Miciah

The image version that is used for Contour/Envoy are taken from an command line arg to the operator, but the operator should be able to derive the information from the Contour-Operator CRD such than different instance of Contour can run their own version requested via the CRD.

Originally posted by @danehans in #44 (comment)

Please describe the problem you have
The operator should only create the certgen Job if the contourcert and envoycert secrets do not exist.

xref: #31

/cc @jpeach @Miciah

We would like to incorporate contour-operator into the Contour release process.

Goals:

• Enable us to make a new projectcontour/contour-operator release shortly after, or in tandem with, each new projectcontour/contour release.
• Keep projectcontour/contour and projectcontour/contour-operator releases in sync as much as possible.
• Automate the contour-operator release process.

To this end, we propose the following changes:

• Define a GitHub action that triggers on each new projectcontour/contour release. This action would open a PR in projectcontour/contour-operator to update config/manager/kustomization.yaml to reference the new Contour image. CI would then automatically verify contour-operator with the new Contour release. The PR description could include instructions for reviewing the PR and completing the projectcontour/contour-operator release.

• Update the Contour release process with an additional/follow-up step to do a manual review of the automatically created PR. If the PR is passing tests with the new Contour release and there are no outstanding issues in projectcontour/contour-operator, a maintainer would merge the PR and follow the instructions in the PR's description to create and push a tag, which would produce a new contour-operator release.

This would be very nice to have for v1alpha1, but it is more a stretch goal than a blocking issue.

As a long-term goal once contour-operator is more mature, it would make sense to use projectcontour/contour-operator to set up Contour in projectcontour/contour CI jobs, but we can track that as a separate issue.

@jpeach, @stevesloka, does this approach seem feasible to you?

Please describe the problem you have
The operator contains contour/envoy image flags that can be set with env vars. It would be helpful if the operator emits one or more log messages that indicate the images used for contour and envoy.

xref: #55

/cc @Miciah

Please describe the problem you have

Add a PR-scoped Github Action to run unit tests and kubebuilder integration tests.

Please describe the problem you have
Make Contour and Envoy network endpoints configurable. For example, I may not want to use a LB service to expose Envoy (default). I may want to only use a ClusterIP or a range of other possibilities for Contour and Envoy.

/cc @jpeach @Miciah

Please describe the problem you have
Should the operator provide the ability to generate and manage a NetworkPolicy based on the configuration (i.e. exposed ports) of a Contour?

xref: https://kubernetes.slack.com/archives/C8XRH2R4J/p1603237175306900?thread_ts=1603237143.306800&cid=C8XRH2R4J

/cc @jpeach @Miciah

Please describe the problem you have
Currently, the Envoy Service is annotated with "service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp" that places the AWS ELB in TCP mode. This causes the ELB to masquerade the client IP. Adding proxy protocol support will allow the client IP to be preserved for ELB'd connections.

Another approach is to use AWS NLB instead of ELB since NLB natively preserves client IPs.

/cc @jpeach @stevesloka @michmike

xref: examples/contour/02-service-envoy

Users may prefer to deploy Envoy using Deployments rather than DaemonSets. Using Deployments provides more control, for example over pod scheduling and over the deployment strategy.

Relates to #60 (/status subresource) and #72 (/scale subresource).

@jpeach, is there background on the decision to use DaemonSets to manage Envoy?

As part of this project, we need to agree on the versioning schema both for the API objects and the software.

I propose initially:

• API versioning uses the Kubernetes versioning standards. This doesn't seem controversial, but writing it down here seems like a net win.
• Software versioning uses semver? Seems like that would make the most sense, given that's what mainline Contour does. A question I would have for everyone in this case is - do we have the operator version match the Contour version? I can see pros and cons there, but I don't want to prejudice the discussion too much.

Please describe the problem you have

• As a cluster admin, I need the ability to run multiple instances of Contour in the same namespace.
• As a cluster admin, I need the ability to run Contour in a namespace other than "projectcontour".

See this Slack thread of additional background.

Please describe the problem you have:

Detect whether gatekeeper is running in the cluster and automatically install the relevant validation configuration for Contour and Envoy.

/cc @jpeach

What steps did you take and what happened:
xref: projectcontour/contour#3058. The port name should be metrics instead of debug:

...
time="2020-10-23T18:38:20Z" level=info msg="started HTTP server" address="0.0.0.0:8000" context=metricsvc
time="2020-10-23T18:38:20Z" level=info msg="started HTTP server" address="127.0.0.1:6060" context=debugsvc

Please describe the problem you have
Add support for managing Envoy through a DaemonSet resource.

Please describe the problem you have
Should the completed pods of the certgen Job be cleaned-up? For example, the completed pods can pile-up:

$ kubectl get po -n projectcontour
NAME                       READY   STATUS        RESTARTS   AGE
contour-57688bcf46-lzr8l   1/1     Running       0          36s
contour-57688bcf46-nrlhn   1/1     Running       0          36s
contour-certgen-7qh79      0/1     Completed     0          119m
contour-certgen-9bkg2      0/1     Completed     0          18h
contour-certgen-9fjwq      0/1     Completed     0          149m
contour-certgen-b28sx      0/1     Completed     0          109m
contour-certgen-gqhcm      0/1     Completed     0          36s
contour-certgen-gwj9z      0/1     Completed     0          25m
contour-certgen-ldbw5      0/1     Completed     0          114m
envoy-x8xbs                0/2     Pending       0          36s

The pods do provide some potentially valuable history:

$ kubectl logs contour-certgen-7qh79 -n projectcontour
Writing "compact" format Secrets to namespace "projectcontour"
secret/contourcert updated
secret/envoycert updated

xref: #36
xref: #34

/cc @jpeach @Miciah

Please describe the problem you have
Currently, RBAC resources are not being compared for equality. For example, a user can change a policy rule for the contour ClusterRole and the operator will not update the ClusterRole to the desired state.

Mirroring what Contour does, we should add the golanglint-ci checks at least as a starting point to help keep code consistent.

Please describe the problem you have
As I work on contour-operator, I’ve been pushing the image to my own repo. An image repo should be created in the Docker Hub projectcontour org for this project.

/assign @jpeach

Update the default Envoy version to 1.16.0.

Originally posted by @skriss in #44 (comment)

Please describe the problem you have
Currently, the contour-certgen is responsible for generating certificates used to secure Contour<>Envoy communication. Cert generation should be configurable, providing options for cert-gen, cert-manager, and contour-operator generated certs.

Additional Considerations:

• Most clusters don't have the necessary feature gate turned on, so the operator could delete the certgen job if it is successful.
• The operator generates certs and stores them in a secret that is referenced by contour/envoy
• Expose cert generation through the Contour API.
• Should the operator run contour certgen binary?

xref: #31

/cc @jpeach @stevesloka @Miciah

Please describe the problem you have
Status of Contour is currently not implemented. Implement a basic level of status that surfaces an available condition based upon the availability of Contour/Envoy pods, etc..

Note: Other conditions should be implemented post v1alpha1.

xref: HTTProxy Status Conditions (Detailed)

/cc @jpeach @Miciah

What steps did you take and what happened:

The operator doesn't manage endpoints so we can remove those from the RBAC config.

What steps did you take and what happened:

We should prefer to use api-machinery semantic equality to compare objects.

https://godoc.org/k8s.io/apimachinery/pkg/api/equality

projectcontour/contour#2995

Please describe the problem you have
Add support for managing Envoy and Contour Service resources.

Please describe the problem you have
Currently a Contour resource can be created in any namespace desired by a user with cluster credentials. A Contour is meant to be a resource exposed to cluster administrators and should be restricted. This issue proposes restricting Contour CRUD operations to the same namespace as the operator (i.e. contour-operator).

/cc @jpeach @stevesloka @Miciah

Please describe the problem you have
Add support for managing Contour's configuration through a ConfigMap resource.

Please describe the problem you have
Currently ensureFoo() funcs simply skip updating a resource if it exists. These funcs should compare the found resource to the desired resource and update said resource if current != desired.

Please describe the problem you have
Evaluate whether standardizing on the stretchr/testify framework for unit tests is worth adding as an external dependency.

/cc @jpeach

cobra is widely used and familiar, generally good enough, and has support for subcommands, generating docs, and other useful features. Let's use cobra for the command-line parsing and so-on (contour-authserver could be an example).

We should update the copyright headers and boilerplate to match the standard Contour one:

// Copyright Project Contour Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

Please describe the problem you have
Enable the status subresource for the contour API:

// +kubebuilder:subresource:status

Please describe the problem you have
projectcontour.io is the home to Contour's official documentation. These docs should be updated to include the operator.

• Operator installation, including any CRDs (if needed). Should the operator be an option in the Getting Started section?
• Deploy an instance of Contour. Should the operator be an option in the Deployment Options section? Note: The uninstall step should include deleting ns/contour-operator when using the operator.
• Contour configuration reference. Similar to Contour's config reference.
• Update the supported versions matrix to include the operator.
• Upgrades- Add content to this or this. Note: Contour will not support upgrades for the v1alpha1 release.
• Generate Contour API reference similar to Contour's API reference docs.
• Update the troubleshooting guide with steps on how to troubleshoot the operator.
• Does the release process doc need to be updated with operator-specific steps/section?
• Update the contributing guide with steps on how to contribute.

/cc @stevesloka @jpeach @Miciah

Please describe the problem you have:

Detect whether prometheus is running in the cluster and automatically install the metrics configuration for Contour and Envoy.

/cc @jpeach

Please describe the problem you have
Envoy containers are statically configured to listen on 80 for http and 443 for https. These ports should be configurable through a Contour resource to provide deployment flexibility. For example, a cluster may have other apps using these common port numbers.

Notes:

• Contour needs to consume these ports for it's --envoy-service-http-port and --envoy-service-https-port config.
• The Envoy service needs to reference the port definitions to configure the targetPort.
• Default to existing 80/443 port definitions.

Please describe the problem you have
Add support for using Operator Lifecycle Manager to manage the operator.

xref Slack discussion: https://kubernetes.slack.com/archives/C8XRH2R4J/p1603824453388600

Please describe the problem you have

Currently, the operator derives the names of the resources needed for a specific contour from a mixture of constants. Adding a single struct to capture all the names and an API to generate this struct from a Contour spec would make the code easier to follow and simplify how resources are created and tracked.

xref #18

Please describe the problem you have
Currently Contour equality is being duplicated across managed dependent resources. Create a pkg that performs the equality matching that can be leveraged across the project.

Add a Github actions to update the main tag at https://hub.docker.com/r/projectcontour/contour-operator after each merge.

We need to verify that CI has the necessary credentials.

Tasks:

• Build & Push :main image
• Build & Push tagged image

xref: #6 (comment)

Please describe the problem you have
Add Gateway API support, specifically GatewayClass and Gateway Support.

xref design doc.

/cc @Miciah @jpeach @youngnick

Please describe the problem you have
The operator logs many events when an operation is being skipped. Although this is nice for early development, it should be removed to reduce noise as the operator matures.

xref: #21 (comment)

/cc @Miciah

Please describe the problem you have
Add support for managing Contour through a Deployment resource.

Please describe the problem you have
Add support for managing Contour certgen through a Job resource.

Please describe the problem you have
The value of the --image flag is currently accepted as an opaque string. The image value should be validated according to the Docker reference type. The same should be accomplished for all image references.

/cc @jpeach

Please describe the problem you have

Add a golangci-lint configuration to make it easier for contributors to match Go style. This reduces the number of nits that need to be addressed in code review.

Please describe the problem you have
When projectcontour/contour#3047 is fixed (targeting v1.11), xds-server-version will become xds-server-versions. Update the configmap to support the pluralization with "v2" being the default and "v3" being a conditional option. Example:

server:
    xds-server-type: contour
    xds-server-versions:     # <--- change to "versions"
    - v2                               # <--- any in the list will be served
    - v3

The github action which pushes a :main tag on each merge to main branch is failing:

@youngnick you hold all the keys to the bot account, can you double check those creds?

Please describe the problem you have
projectcontour/contour has several labels that are applicable to the operator. Replicate these labels.

xrefs:

• https://github.com/projectcontour/contour/blob/main/site/_data/github-labels.yaml
• https://github.com/projectcontour/contour/blob/main/.github/workflows/sync-labels.yaml

The contour CRD should define the /scale subresource to enable the use of tools such as kubectl scale and HPA with contour CRs.

The CRD needs to provide the following information under its status field:

• Label selector for the scalable resource (Contour pods).
• Number of available replicas.

/scale therefore depends on the /status subresource (xref: #60).

The CRD also needs to provide the following under its spec field:

• Number of desired replicas.

Once this information is available, the CRD can define the /status subresource as follows:

    subresources:
      scale:
        labelSelectorPath: .status.selector
        specReplicasPath: .spec.replicas
        statusReplicasPath: .status.contourReplicasAvailable

Post-v1alpha1.

Please describe the problem you have
Add support for managing Contour CRDs (i.e. HTTPProxy) through a CustomResourceDefinition resource.

/cc @jpeach @Miciah