Comments (6)
Terraform plan indicates that the missing resources are for ASGs:
# module.eks_blueprints_addons.module.aws_node_termination_handler.aws_iam_policy.this[0] will be created
+ resource "aws_iam_policy" "this" {
+ arn = (known after apply)
+ description = "IAM Policy for AWS Node Termination Handler"
+ id = (known after apply)
+ name = (known after apply)
+ name_prefix = "aws-node-termination-handler-"
+ path = "/"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "ec2:DescribeInstances",
+ "autoscaling:DescribeTags",
+ "autoscaling:DescribeAutoScalingInstances",
]
+ Effect = "Allow"
+ Resource = "*"
},
+ {
+ Action = "autoscaling:CompleteLifecycleAction"
+ Effect = "Allow"
},
from terraform-aws-cloudbees-ci-eks-addon.
The resources is taken from variable aws_node_termination_handler_asg_arns
, which should be populated along with enable_aws_node_termination_handler
For example you could do:
enable_aws_node_termination_handler = true
aws_node_termination_handler_asg_arns = data.aws_autoscaling_groups.eks_node_groups.arns
where we took the ASGs via data source:
data "aws_autoscaling_groups" "eks_node_groups" {
depends_on = [ module.eks ]
filter {
name = "tag-key"
values = ["eks:cluster-name"]
}
}
from terraform-aws-cloudbees-ci-eks-addon.
I reponed this issue, the proposal by @wellsiau-aws works well for apply
but not for destroy
. The following error appears
│ Error: Invalid for_each argument
│
│ on .terraform/modules/eks_blueprints_addons/main.tf line 1547, in resource "aws_autoscaling_lifecycle_hook" "aws_node_termination_handler":
│ 1547: for_each = { for k, v in var.aws_node_termination_handler_asg_arns : k => v if var.enable_aws_node_termination_handler }
│ ├────────────────
│ │ var.aws_node_termination_handler_asg_arns is a list of string, known only after apply
│ │ var.enable_aws_node_termination_handler is true
│
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this
│ resource.
│
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.
It leaves the tf files/project into a stuck state. It is not possible now apply
either
Looking into the terraform state files the requested data is there
{
"mode": "data",
"type": "aws_autoscaling_groups",
"name": "eks_node_groups",
"provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
"instances": [
{
"schema_version": 0,
"attributes": {
"arns": [
"arn:aws:autoscaling:us-east-1:324005994172:autoScalingGroup:02142c6d-ed16-44cd-886f-002bd6e6b33d:autoScalingGroupName/eks-mg_cbApps-2024010915313845610000002a-42c6776c-e2ee-4cc3-77e9-8256bceb1b51",
...
"arn:aws:autoscaling:us-east-1:324005994172:autoScalingGroup:fe9e0cec-1048-4619-993a-c177047ec420:autoScalingGroupName/eks-mg_k8sApps_1az-20240109153138455200000026-72c6776c-e2ed-f4d6-1f8b-77d268987c2a"
],
"filter": [
{
"name": "tag-key",
"values": [
"eks:cluster-name"
]
}
],
"id": "us-east-1",
"names": [
"eks-cbc-aaaaaaaaaa-eks-node-group-v1-xxxxxxxxxxxx",
...
"eks-cbc-bbbbbbbbbbb-eks-node-group-v2-xxxxxxxxxxxx",
]
},
"sensitive_attributes": []
}
]
},
The issue got solved by commenting aws_node_termination_handler_asg_arns
and then it could be performed a destroy
enable_aws_node_termination_handler = false
#aws_node_termination_handler_asg_arns = data.aws_autoscaling_groups.eks_node_groups.arns
from terraform-aws-cloudbees-ci-eks-addon.
Looking at the Complete test case https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/blob/main/tests/complete/main.tf seems that right configuration is only valid with self_managed_node_groups
which contains autoscaling_group_arn
as output. eks_managed_node_groups
submodule does not.
enable_aws_node_termination_handler = true
aws_node_termination_handler_asg_arns = [for asg in module.eks.self_managed_node_groups : asg.autoscaling_group_arn]
from terraform-aws-cloudbees-ci-eks-addon.
good catch, I was taking shortcut earlier and after diving deeper I saw a few limitation.
it boils down to the availability of the managed node group Auto scaling gorup ARNs to be populated deterministically instead of using data source.
looking at the [EKS module] I can see that only the ASG name is available as the output
Further down, the aws_eks_node_group resource itself does not have ASG arns as the reference, this itself boils down to the EKS API itself
from terraform-aws-cloudbees-ci-eks-addon.
It can not be implemented for Managed Node Groups
from terraform-aws-cloudbees-ci-eks-addon.
Related Issues (20)
- [CI, CloudBees plataform] Replace Credentials by Configuring CloudBees OIDC with AWS
- [CI, GHA] Terraform.yaml HOT 1
- [Blueprints, 02-at-scale]: Velero: Complete Recommendation from CloudBees
- [Blueprints, 02-at-scale]: Casc: Install Jenkins Health Advisor
- [Blueprints, 02-at-scale]: Casc: Credentail Example
- [Blueprints, 02-at-scale] Velero deployment fails HOT 3
- [Blueprints, 02-at-scale] KMS Encryption
- [All] Adding Issue Templates HOT 1
- [CI] Terraform Randomly fails with creating KMS Alias (alias/eks/cbci-bpxx-ci-xx-eks): AlreadyExistsException HOT 3
- [Blueprints, 02-at-scale] ALB stickiness enabled on OC HOT 5
- [Blueprints, 01-getting-started]: Setting gp3 as default Storage Class
- [Blueprints, all] Automate PR with version update HOT 2
- [Blueprints, 02-at-scale]: Casc not working for variables at item level
- [CI] Investigate Terraform Test as alternative for full rest cycle
- [Blueprints, 02-at-scale]: Openldap: Use jp-gouin.github.io/helm-openldap
- [CI] Run blueprints only tf files are updated
- [Doc] Documented simplest example contains wrong variable names HOT 1
- [Doc] Add Full URL for the Main Read Page
- [Blueprints, 02-at-scale]: Kube-Prometheus-Stack: Adding Modern Dashboard for Exploding Node Exporter Data in K8s
- [Blueprints, 02-at-scale]: EKS 1.28 fails HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terraform-aws-cloudbees-ci-eks-addon.