GCP subnet share conditions not working correctly about terraform-google-project-factory HOT 7 CLOSED

When variable vpc_networks is incorrectly formed the failure happens and leaves all subnets shared
Correct formatting and functional
shared_vpc_subnets: '["projects/vpc-abc/regions/europe-west3/subnetworks/subnet1"]'

Incorrect formatting and error observed below
shared_vpc_subnets: '["projects/vpc-abc/regions/europe-west3/subnet1"]'

* google_compute_subnetwork_iam_member.service_account_role_to_vpc_subnets: Error retrieving IAM policy for Compute Subnetwork non-prod-host/europe-west3/projects: googleapi: Error 404: The resource 'projects/non-prod-host/regions/europe-west3/subnetworks/projects' was not found, notFound

* module.core-factory.google_compute_subnetwork_iam_member.apis_service_account_role_to_vpc_subnets: 1 error(s) occurred:

* google_compute_subnetwork_iam_member.apis_service_account_role_to_vpc_subnets: Error retrieving IAM policy for Compute Subnetwork non-prod-host/europe-west3/projects: googleapi: Error 404: The resource 'projects/non-prod-host/regions/europe-west3/subnetworks/projects' was not found, notFound

from terraform-google-project-factory.

#97 is related.

from terraform-google-project-factory.

To add some more context here, these IAM resources are using a split() on the shared_vpc_subnets array to populate the subnetwork attribute on the subnet IAM member resources: https://github.com/terraform-google-modules/terraform-google-project-factory/blob/master/modules/core_project_factory/main.tf#L219-L265 If you pass in an element to that array that is't properly formatted with projects/<project_name>/regions/<region>/subnetworks/<subnetwork> then the element() function will return an empty string, and that's what gets passed to the IAM membership resource. Passing an empty string is the same as if you didn't pass it, and so that's why permissions are granted at the project and not the subnetwork level.

This will also happen if you pass the subnets_self_links output to var.shared_vpc_subnets (which is what this PR is referencing: terraform-google-modules/terraform-google-network#33 ).

If we're going to continue to use the projects/<project_name>/regions/<region>/subnetworks/<subnetwork> format we might want to check for the strings projects, regions, and subnetworks being in the correct places for each element in the array as something of a preflight check?

from terraform-google-project-factory.

@aaron-lane @morgante
how do you think what is the best option for validation, we have 2 possible points:

1. terraform-google-project-factory/modules/core_project_factory/scripts/preconditions/preconditions.py, which is not good option as for me
2. do it in terraform module, here is the code I would solve this:

variable "shared_vpc_subnets" {
  description = "List of subnets fully qualified subnet IDs (ie. projects/$project_id/regions/$region/subnetworks/$subnet_id)"
  type        = "list"
  default     = [
    "http://projects/base-project-196723/regions/us-east1/subnetworks/default",
    "projects/base-project-196723/regions/us-central1/subnetworks/default",
    "XXXX/base-project-196723/regions/us-central1/subnetworks/subnet-1",
  ]
}

resource "null_resource" "invalid_subnets" {
  count = "${length(var.shared_vpc_subnets)}"
  triggers = {
    subnet  = "${replace(var.shared_vpc_subnets[count.index],
    "/(https://www.googleapis.com/compute/v1/)?projects/[a-z0-9-]+/regions/[a-z0-9-]+/subnetworks/[a-z0-9-]+/", "") == "" ? false : true}"
  }
}

locals {
    invalid_subnets  = "${null_resource.invalid_subnets.*.triggers.subnet}"
}

resource "null_resource" "valid_subnet" {
  count = "${length(var.shared_vpc_subnets)}"
  triggers = {
    subnet = "${local.invalid_subnets[count.index] ? "dummy-subnet" : var.shared_vpc_subnets[count.index]}"
  }
}

output "valid_subnets" {
  value       = "${null_resource.valid_subnet.*.triggers.subnet}"
}

It will give following output:

valid_subnets = [
    dummy-subnet,
    projects/base-project-196723/regions/us-central1/subnetworks/default,
    dummy-subnet
]

which cause fail for incorrect subnet item

from terraform-google-project-factory.

@kopachevsky this should indeed be solved in the Terraform configuration; preconditions.py is used to verify the presence of required permissions.

If we are relying on the google_subnet_iam_member resources to throw errors when invalid subnets are provided then we should choose a more descriptive string than simply "dummy-subnet". Alternatively, we could use one of the solutions from this thread to raise a descriptive error.

from terraform-google-project-factory.

@aaron-lane I agree "dummy-subnet" not good way to point user with the problem it his config, I've tried options form issue you mentioned above initially, for some reason it wont work for me, I even raised stackoverflow question https://stackoverflow.com/questions/56042077/terraform-v0-11-xx-null-resource-not-always-works-as-assertion

from terraform-google-project-factory.

@kopachevsky As an incremental fix, please provide a PR which only automatically strips the https://www.googleapis.com/compute/v1/ prefix. This should be very straightforward and would immediately fix terraform-google-modules/terraform-google-network#33

from terraform-google-project-factory.