"aws-auth" is forbidden: User "system:anonymous" cannot get resource about terraform-aws-eks-cluster HOT 8 CLOSED

Config map access by the terraform user who makes the cluster is enabled by default, and in fact cannot be disabled. We explain in the README and in more detail in the Release Notes for v0.42.0 the various problems with authenticating to the Kubernetes cluster.

Most likely what happened to you is that Terraform got and cached the auth token from the data source, but it expired while it was in the cache and therefore authentication failed when you tried to apply the changes. With kube_exec_auth_enabled = true you get a fresh token every time you run plan or apply.

from terraform-aws-eks-cluster.

@christoffer-eide thanks for sharing. That fixed my issue when I was trying to set public_access_cidrs.

I have Public and Private endpoint set but I can only communicate via the Public endpoint. If I try to communicate with the Private endpoint (e.g. from a VM build in the same VPC like the EKS cluster) I get no response (I can resolve IP but no connectivity).

Any idea what could be wrong?

try to probe with telnet to confirm that you can reach TCP of that address, otherwise it could be just the routing tables issue (in VPC subnets) or security groups, can't think of anything else that might restrict it. unless the internal endpoint has additional mechanisms - could it restrict access over the network based on IAM role? I know it does for OpenSearch (happened when I was setting up logging earlier this year, so IAM role policy that allows access to service was mandatory, and it somehow controlled the actual networking access to the endpoint addresses, I have no idea how it works behind the scenes but it's dynamic based on IAM. Might be same with EKS then? )

from terraform-aws-eks-cluster.

Thanks @Dmitry1987, I have found out the issue. I didn't associate the default VPC SG and VPN CIDR via:

  allowed_security_group_ids = [module.vpc.vpc_default_security_group_id]
  allowed_cidr_blocks        = [module.vpc.vpc_cidr_block]

from terraform-aws-eks-cluster.

Try setting kube_exec_auth_enabled = true in your eks_cluster module

from terraform-aws-eks-cluster.

thank you, will try that. but why does this setting cause the error, and what else happens when i disable the exec auth flag?

from terraform-aws-eks-cluster.

kube_exec_auth_enabled just controls if the authentication token should be fetched from the terraform data source aws_eks_cluster_auth, or by executing the command eks get-token.

I've had problems in the past where the aws_eks_cluster_auth didn't return a token when I tried to modify the cluster

from terraform-aws-eks-cluster.

@christoffer-eide thanks for sharing. That fixed my issue when I was trying to set public_access_cidrs.

I have Public and Private endpoint set but I can only communicate via the Public endpoint. If I try to communicate with the Private endpoint (e.g. from a VM build in the same VPC like the EKS cluster) I get no response (I can resolve IP but no connectivity).

Any idea what could be wrong?

from terraform-aws-eks-cluster.

Config map access by the terraform user who makes the cluster is enabled by default, and in fact cannot be disabled. We explain in the README and in more detail in the Release Notes for v0.42.0 the various problems with authenticating to the Kubernetes cluster.

Most likely what happened to you is that Terraform got and cached the auth token from the data source, but it expired while it was in the cache and therefore authentication failed when you tried to apply the changes. With kube_exec_auth_enabled = true you get a fresh token every time you run plan or apply.

v3.0.0 kube_exec_auth_enabled = true
The problem still exists

from terraform-aws-eks-cluster.