Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
What is the outcome that you are trying to reach?
Hi, I'd like to add the ability to install the ACK Amazon Elastic Kubernetes Service controller.
This would allow users to create Amazon Elastic Kubernetes Service resources (Cluster, Addon, NodeGroup, FargateProfile) with ACK.
Describe the solution you would like
Nothing fancy, following the same pattern as the other controllers is the way.
Additional context
I'm using my fork of terraform-aws-eks-ack-addons with the terraform-aws-eks-blueprints.
You can see it's changes here, but I'll outline them in this issue too.
module "eks_blueprints_ack_addons" {
source = "github.com/pdemagny/terraform-aws-eks-ack-addons?ref=feat-eks-support"
cluster_id = module.eks_blueprints.eks_cluster_id
# Wait for data plane to be ready
data_plane_wait_arn = module.eks_blueprints.managed_node_group_arn[0]
enable_api_gatewayv2 = false
enable_dynamodb = false
enable_s3 = false
enable_rds = false
enable_amp = false
enable_eks = true
tags = local.tags
}This controller requires to pay more attention to the iRSA settings as no single pre-existing IAM Policy covers the permissions needed to create all its resources.
The recommended inline policy from ACK is not enough either ... If you want to create NodeGroups or FargateProfiles, you need more permissions.
Hence the addition of the inline policies.
Here is what I've come up with after testing every resources:
################################################################################
# Elastic Kubernetes Service
################################################################################
locals {
eks_name = "ack-eks"
}
module "eks" {
source = "github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon?ref=v4.12.2"
count = var.enable_eks ? 1 : 0
helm_config = merge(
{
name = local.eks_name
chart = "eks-chart"
repository = "oci://public.ecr.aws/aws-controllers-k8s"
version = "v0.1.7"
namespace = local.eks_name
create_namespace = true
description = "ACK eks Controller v2 Helm chart deployment configuration"
values = [
# shortens pod name from `ack-eks-eks-chart-xxxxxxxxxxxxx` to `ack-eks-xxxxxxxxxxxxx`
<<-EOT
nameOverride: ack-eks
EOT
]
},
var.eks_helm_config
)
set_values = [
{
name = "serviceAccount.name"
value = local.eks_name
},
{
name = "serviceAccount.create"
value = false
},
{
name = "aws.region"
value = local.region
}
]
irsa_config = {
create_kubernetes_namespace = true
kubernetes_namespace = try(var.eks_helm_config.namespace, local.eks_name)
create_kubernetes_service_account = true
kubernetes_service_account = local.eks_name
irsa_iam_policies = [aws_iam_policy.ack_eks_policy[0].arn, data.aws_iam_policy.eks[0].arn]
}
addon_context = local.addon_context
}
resource "aws_iam_policy" "ack_eks_policy" {
count = var.enable_eks ? 1 : 0
name = "${local.cluster_id}-ack-eks-sa-policy"
description = "IAM policy for ${local.eks_name} Service Account"
path = "/"
policy = data.aws_iam_policy_document.ack_eks_policy_document[0].json
tags = local.tags
}
data "aws_iam_policy_document" "ack_eks_policy_document" {
count = var.enable_eks ? 1 : 0
statement {
sid = "ACKEKSPolicy1" # Recommended ACK inline Policy, see https://github.com/aws-controllers-k8s/eks-controller/blob/main/config/iam/recommended-inline-policy
effect = "Allow"
actions = ["eks:*"]
resources = ["*"]
}
statement {
sid = "ACKEKSPolicy2" # iam:GetRole is required to create NodeGroups and iam:CreateServiceLinkedRole is required to create FargateProfiles
effect = "Allow"
actions = [
"iam:GetRole",
"iam:CreateServiceLinkedRole"
]
resources = ["*"]
}
statement {
sid = "ACKEKSPolicy3" # Required to create NodeGroups
effect = "Allow"
actions = ["iam:PassRole"]
resources = ["*"]
condition {
test = "StringEquals"
variable = "iam:PassedToService"
values = ["eks.amazonaws.com"]
}
}
}
data "aws_iam_policy" "eks" {
count = var.enable_eks ? 1 : 0
name = "AmazonEKSServicePolicy"
}Here are my test results:
EKS
Cluster
apiVersion: eks.services.k8s.aws/v1alpha1
kind: Cluster
metadata:
name: my-ack-test-cluster
spec:
name: my-ack-test-cluster
roleARN: arn:aws:iam::<REDACTED>:role/crossplane-ack-meetup-cluster-role
resourcesVPCConfig:
endpointPrivateAccess: true
endpointPublicAccess: true
subnetIDs:
- "subnet-02421b2bc404c9324"
- "subnet-0604d52bdcb46e8b6"
2022-12-01T13:37:10.936Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Cluster", "namespace": "default", "name": "my-ack-test-cluster", "is_adopted": false, "generation": 1}
Addon
apiVersion: eks.services.k8s.aws/v1alpha1
kind: Addon
metadata:
name: vpc-cni
spec:
name: vpc-cni
addonVersion: "v1.12.0-eksbuild.1"
clusterName: my-ack-test-cluster
resolveConflicts: "OVERWRITE"
---
apiVersion: eks.services.k8s.aws/v1alpha1
kind: Addon
metadata:
name: coredns
spec:
name: coredns
addonVersion: "v1.8.7-eksbuild.3"
clusterName: my-ack-test-cluster
resolveConflicts: "OVERWRITE"
---
apiVersion: eks.services.k8s.aws/v1alpha1
kind: Addon
metadata:
name: kube-proxy
spec:
name: kube-proxy
addonVersion: "v1.23.13-eksbuild.2"
clusterName: my-ack-test-cluster
resolveConflicts: "OVERWRITE"
2022-12-01T15:16:11.236Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Addon", "namespace": "default", "name": "vpc-cni", "is_adopted": false, "generation": 1}
2022-12-01T15:16:11.950Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Addon", "namespace": "default", "name": "coredns", "is_adopted": false, "generation": 1}
2022-12-01T15:16:12.802Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Addon", "namespace": "default", "name": "kube-proxy", "is_adopted": false, "generation": 1}
NodeGroup
apiVersion: eks.services.k8s.aws/v1alpha1
kind: Nodegroup
metadata:
name: my-ack-test-ng
spec:
name: my-ack-test-ng
clusterName: my-ack-test-cluster
subnets:
- "subnet-02421b2bc404c9324"
- "subnet-0604d52bdcb46e8b6"
nodeRole: arn:aws:iam::<REDACTED>:role/crossplane-ack-meetup-cluster-role
scalingConfig:
minSize: 1
maxSize: 1
desiredSize: 1
2022-12-01T15:23:06.006Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Nodegroup", "namespace": "default", "name": "my-ack-test-ng", "is_adopted": false, "generation": 1}
2022-12-01T15:51:13.894Z INFO ackrt deleted resource {"account": "", "role": "", "region": "eu-west-1", "kind": "Nodegroup", "namespace": "default", "name": "my-ack-test-ng", "generation": 3}
FargateProfile
❯ cat pod-execution-role-trust-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:eks:eu-west-1:<REDACTED>:fargateprofile/my-ack-test-cluster/*"
}
},
"Principal": {
"Service": "eks-fargate-pods.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
❯ aws iam create-role \
--role-name AmazonEKSFargatePodExecutionRole \
--assume-role-policy-document file://"pod-execution-role-trust-policy.json"
❯ aws iam attach-role-policy \
--policy-arn arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy \
--role-name AmazonEKSFargatePodExecutionRoleapiVersion: eks.services.k8s.aws/v1alpha1
kind: FargateProfile
metadata:
name: my-ack-test-profile
spec:
name: my-ack-test-profile
clusterName: my-ack-test-cluster
podExecutionRoleARN: arn:aws:iam::<REDACTED>:role/AmazonEKSFargatePodExecutionRole
subnets:
- "subnet-087c10af4f1bc624b"
- "subnet-0f29941bb08e3c58a"
selectors:
- namespace: default
2022-12-01T21:07:24.631Z INFO ackrt created new resource {"account": "", "role": "", "region": "eu-west-1", "kind": "FargateProfile", "namespace": "default", "name": "my-ack-test-profile", "is_adopted": false, "generation": 1}
2022-12-01T21:11:29.696Z INFO ackrt deleted resource {"account": "", "role": "", "region": "eu-west-1", "kind": "FargateProfile", "namespace": "default", "name": "my-ack-test-profile", "generation": 2}
In the end, the created NodeGroup can't join the created Cluster because of the lacking RBAC mapping from the missing aws-auth ConfigMap, and the missing security group rules.
But stricly on the controller side, the create & delete actions are allowed with this set of IAM permissions.
I'm proposing this changes in #34
I'm also going to propose the change in recommended iam policy upstream.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent