Failed to pull image in bootstrap cluster. certificate signed by unknown authority. Custom CA about eks-anywhere HOT 6 CLOSED

Hello @pulberg, thank you for submitting the issue. Does your corp environment have access to https://public.ecr.aws?

from eks-anywhere.

Yes, these pulled correctly -

public.ecr.aws/eks-anywhere/cli-tools:v0.1.0-eks-a-1
public.ecr.aws/eks-anywhere/kubernetes-sigs/kind/node:v1.21.2-eks-d-1-21-4-eks-a-1

I also tested pulling the cert-manager images manually and that works, so there is something happening in the setup of the cluster that is not happy with the corp CA. Even after pulling the images manually the x509 errors persist.

from eks-anywhere.

The cert manager image, along with a number of others, are pulled inside the bootstrap kind cluster whereas those other two you mentioned are pulled on the host machine. The host must have the trusted certs /ca whereas the kind cluster does not.

Not sure the best way to handle this but we can do some looking around. One option would be to do all pulling on the host and use kind's load image ability to load them into the running bootstrap container.

from eks-anywhere.

The cert manager image, along with a number of others, are pulled inside the bootstrap kind cluster whereas those other two you mentioned are pulled on the host machine. The host must have the trusted certs /ca whereas the kind cluster does not.

kubernetes-sigs/kind#941 documents this error and provides a workaround to add the trusted cert/ca to the kind cluster. Kind does offer also the ability to mount volumes with extraMounts in the config kubernetes-sigs/kind#1010.

Not sure the best way to handle this but we can do some looking around. One option would be to do all pulling on the host and use kind's load image ability to load them into the running bootstrap container.

All images needed by the kind cluster could be pulled from the workstation and archived. Then it would be a simple load of the archive to kind cluster. Ref: https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster.

from eks-anywhere.

Thanks @antoniordz96!

The more I think about it the more I'm not sure about the pulling images on the host approach. That would def solve this exact problem, but if you were creating a vsphere cluster the kind cluster would be need to be able to make calls to the vcenter api, would those also be being intercepted like this and need the CA? Technically for vcenter we can ignore cert failures by providing the thumbprint or just ignoring tls, but I wonder if in a general sense we would just be kicking the problem down the line in the process. If you were standing up a vsphere cluster, would this custom CA be something you would want to exist in your cluster in case services are making calls to other internal services using this CA?

from eks-anywhere.

There has been no activity on this issue for 60 days. Labeling as stale and closing in 7 days if no further activity.

from eks-anywhere.