Removing the concierge secret makes ingress-nginx reload fail about pinniped HOT 2 CLOSED

Hi @aslafy-z,

Thanks for creating an issue! I assume you are using an ingress for the Concierge's impersonation proxy port?

As you've noted, there are certain TLS certs that are auto-created and auto-rotated. Be careful here though, the api.servingCertificate.durationSeconds controls the rotation of a different auto-generated TLS certificate. That cert is used to help register the Concierge as an aggregated API server with the Kubernetes API server (on the APIService resource). That cert is only used inside the cluster between the Kubernetes API server and the Concierge, and should not be used for the impersonation proxy (which serves traffic from clients that are outside the cluster). You should not need to use an ingress controller for this, because the installation manifest yaml will create a ClusterIP service for this purpose, and it should only receive traffic from inside the cluster (more specifically only from the Kubernetes API server).

As you've seen, the default behavior of the impersonation proxy is to create its own CA, create its own TLS certificate, create a load balancer service, and start serving traffic using the IP of the load balancer service as the ingress address for clients. This auto-generated TLS certificate has a 100 year expiration and will not be rotated by default. (It is created by this controller.)

The default behavior is probably fine for simple use cases, but for Concierge admins who have more complex networking/ingress requirements it may not be sufficient. For example, if you need to use an ingress for all incoming traffic to the cluster, then you might not want to have a separate load balancer for the impersonation proxy.

That's why you are able to change the service that the Concierge will create, if you do not prefer to use a load balancer. You can tell it to create a ClulsterIP service instead of an LB service, or you can tell it not to create a service at all because you prefer to create your own. You can configure this in the CredentialIssuer.spec (see docs).

In recent versions of Pinniped, there is a new feature where you can now also control the TLS certs to be served by the impersonation proxy. (See blog post for a detailed example of how to use this.) This, together with being able to create your own service, gives you almost complete control over ingress and TLS works for your impersonation proxy on each cluster.

Please let us know if that explanation helps, or if you have any more questions about setting up ingress for the Concierge.

from pinniped.

Hi @aslafy-z, did the above comment help? I didn't hear back from you so I will close this issue for now, but please feel free to add questions/comments or reopen the issue.

from pinniped.