cloudposse / terraform-aws-vpc-peering-multi-account Goto Github PK

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Describe the Feature

Support for multi-tenant peering / peering that spans projects, regions, environments, etc.

The naming convention created by https://registry.terraform.io/modules/cloudposse/label/null/latest -- in the way that has been set up in this repo -- it is assumed these are single tenant.

I am happy to make a pull request for this, as I've done this myself, but I'm not a complete terraform export and am not as familiar with your testing requirements / conventions as you might like.

Expected Behavior

Separately named resources for requester/accepter depending on project

Use Case

I have infrastructure that is shared between multiple projects, regions, and environments. One of these resources is our database VPC, which needs to peer with stage, prod, and a test resource (due to some limitations of instance sizes, pricing, and availability, it is a multi-tenant DB)

Describe Ideal Solution

If there was an option of setting:

• namespace
• environment
• stage
• name
• delimiter
• attributes
• tags

for both requester and accepter, that would be very useful, as these are not always the same.
I can see supporting the current behavior and the new feature by:

1. Create module.requester_context and module.accepter_context separately instead of using the module.this
2. use the var.requester_X / var.accepter_X first, then default to var.X for the module.accepter_context / module.requester_context
3. use module.requester_context and module.accepter _context instead of your current module.this

that should allow for this repo to support multi-tenant VPC peering, or projects that need to peer between projects/regions/etc.

Alternatives Considered

Set values that expand to both resources, however that breaks tag lookup

Additional Context

Add any other context or screenshots about the feature request here.

Describe the Bug

when running enabled = false, terraform doesn't assume roles on account of the defined assume_role variable as it is not for_eached: [edit, garbled]

https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account/blob/82ba0a4b1612aa7b5e93c188aeb799e1640601a5/requester.tf#L67C5-L67C88

as a result, teardown is not possible if using either of:

• requester_aws_assume_role_arn
• accepter_aws_assume_role_arn

this causes: Error: reading EC2 VPC Peering Connection (pcx-vpc_peering_id): UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::0123456789:assumed-role/some_role is not authorized to perform: ec2:DescribeVpcPeeringConnections because no identity-based policy allows the ec2:DescribeVpcPeeringConnections action

Expected Behavior

setting enabled = false should plan/teardown successfully when using requester_aws_assume_role_arn | accepter_aws_assume_role_arn

Steps to Reproduce

• start multi-account peering with requester_aws_assume_role_arn or accepter_aws_assume_role_arn
• apply
• set enabled = false
• plan -> permission denied (assuming the role terraform is running as initially doesn't have permissions to either of the two accounts - and would depend on role chaining)

Screenshots

No response

Environment

No response

Additional Context

I believe the provider should always be enabled, including the role_arn part, so removing the enabled ternary part:

for_each = local.enabled && var.accepter_aws_assume_role_arn != "" ? ["true"] : []

to ->

for_each = var.[accepter|requester]_aws_assume_role_arn
should fix the issue. will likely draft a pr later.

Hey there, I am using your terraform-aws-vpc module in combination with this one to create and peer a vpc. I'm having what seems it may be an order of operations issue? Here is my main.tf:

module "vpc" {
  source     = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=0.4.0"
  namespace  = "${var.namespace}"
  stage      = "${var.stage}"
  name       = "${var.app_name}"
  cidr_block = "${var.cidr_block}"
}

data "aws_availability_zones" "available" {}

module "dynamic_subnets" {
  source             = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=0.4.0"
  namespace          = "${var.namespace}"
  stage              = "${var.stage}"
  name               = "${var.app_name}"
  region             = "${var.region}"
  availability_zones = ["${data.aws_availability_zones.available.names}"]
  vpc_id             = "${module.vpc.vpc_id}"
  igw_id             = "${module.vpc.igw_id}"
  cidr_block         = "${var.cidr_block}"
}

module "vpc_peering" {
  source           = "git::https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account.git?ref=0.4.0"
  namespace        = "${var.namespace}"
  stage            = "${var.stage}"
  name             = "${var.app_name}"
  auto_accept      = true

  accepter_aws_assume_role_arn             = "${var.vpn_account_arn}"
  accepter_region                          = "${var.vpn_region}"
  accepter_vpc_id                          = "${var.vpn_vpc_id}"
  accepter_allow_remote_vpc_dns_resolution = "true"

  requester_aws_assume_role_arn             = "${var.account_arn}"
  requester_region                          = "${var.region}"
  requester_vpc_id                          = "${module.vpc.vpc_id}"
  requester_allow_remote_vpc_dns_resolution = "true"
}

When I run terraform plan I am getting the following error:

Error: Error refreshing state: 1 error(s) occurred:

* module.vpc_peering.data.aws_route_table.requester: data.aws_route_table.requester: value of 'count' cannot be computed

My only guess as to what could be happening here is it's failing to get the route table data because the vpc does not yet exist? You'd think it would fail with an aws error rather than an error on the counter though. Hopefully someone might have some insight here.

I was able to get this working by splitting the peering into it's own tf file and running that after this one creates the vpc and passing forward the tfstate file. But ideally I'd like it all to share a tfstate file otherwise if I need to update this file it will wipe out the peering connection configuration.

Thanks

Describe the Bug

The module is unable to support for_each since it contains provider configuration:

Module "foo" cannot be used with for_each because it contains a
nested provider configuration for "aws.requester", at
.terraform/modules/foo/requester.tf:60,10-15.

Expected Behavior

The module should support for_each loops.

Additional Context

I believe passing the providers into the module (instead of configuring the providers w/in the module) would do the trick. I labeled this as bug but it may be a feature request instead.

Describe the Bug

If one of the subnets in one of the VPCs is using the default "main" routing table, the module fails.

Steps to Reproduce

Steps to reproduce the behavior:

1. Create a new VPC
2. Create new subnets in the VPC without associating route tables with them
3. Use the VPC as the "accepter"
4. See error

Error: query returned no results. Please change your search criteria and try again

  on .terraform/modules/vpc_peering/accepter.tf line 64, in data "aws_route_table" "accepter":
  64: data "aws_route_table" "accepter" {

Additional Context

This bug was introduced by #44 which incorporated #31 but still did not really fix #30.

I propose eliminating the option to filter subnets because it never really worked. If someone wants it back, they can submit a PR to implement it properly.

Problems with filtering by subnet

Problem 1: Finding the route tables to modify

Sub-problem 1A: Looking up the route table for a subnet

You cannot find the route table for a subnet via

data "aws_route_table" "subnet" {
  subnet_id = local.subnet_id
}

because this will throw an error if the subnet does not have an associated route table and is instead using the default "main" table. That is the source of this bug.

Because there is no way to make this conditional, we simply cannot use data "aws_route_table".

Sub-problem 1B: Looking up the subnet for a route table

You could try looking up all the route tables and filtering them by associated subnets, but

• this is hard, because you have to treat the "main" route table specially and match it to the subnets that have no associated route table, and
• what do you do if you one route table is associated with both an included and an excluded subnet?

Conclusion: you might as well add routes to all the route tables, because you cannot promise better restrictions

After all, the entire VPC might have only one, default route table.

Problem 2: Why are we looking up subnets by tag, anyway?

The only meaningful restriction we can make is to limit routing to some portion of the VPC's CIDR, so we might as well pass explicit CIDRs for each VPC and just route those in all routing tables. It is not at all clear that selecting the CIDRs by subnet tag is more convenient than explicitly specifying them (or having the root module look them up any way it wants to).

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Describe the Feature

Docs on how to create this https://github.com/cloudposse/terraform-aws-components/tree/master/modules/vpc-peering

It would be nice to create the accepter_aws_assume_role_arn IAM role using a submodule

locals {
  account_id = data.aws_caller_identity.current.account_id
}

data "aws_caller_identity" "current" {}

data "aws_iam_policy_document" "vpc_peering" {
  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["arn:aws:ec2:*:${var.accepter_account}:route-table/*"]

    actions = [
      "ec2:CreateRoute",
      "ec2:DeleteRoute",
    ]
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]

    actions = [
      "ec2:DescribeVpcPeeringConnections",
      "ec2:DescribeVpcs",
      "ec2:ModifyVpcPeeringConnectionOptions",
      "ec2:DescribeSubnets",
      "ec2:DescribeVpcAttribute",
      "ec2:DescribeRouteTables",
    ]
  }

  statement {
    sid    = ""
    effect = "Allow"

    resources = [
      "arn:aws:ec2:*:${var.accepter_account}:vpc-peering-connection/*",
      "arn:aws:ec2:*:${var.accepter_account}:vpc/*",
    ]

    actions = [
      "ec2:AcceptVpcPeeringConnection",
      "ec2:DeleteVpcPeeringConnection",
      "ec2:CreateVpcPeeringConnection",
      "ec2:RejectVpcPeeringConnection",
    ]
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["arn:aws:ec2:*:${var.accepter_account}:vpc-peering-connection/*"]

    actions = [
      "ec2:DeleteTags",
      "ec2:CreateTags",
    ]
  }
}

resource "aws_iam_role" "vpc_peering" {
  name               = "vpc-peering-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Sid    = ""
        Principal = {
          AWS = [
            "arn:aws:iam::${var.requester_account}:root",
          ]
        }
      },
    ]
  })
}

resource "aws_iam_role_policy" "vpc_peering" {
  name = "vpc-peering-policy"
  role = aws_iam_role.vpc_peering.id

  policy = data.aws_iam_policy_document.vpc_peering.json
}

The way we have set up our accounts is we have one auth account and then every other account in our org has an assume role that the auth account can assume. This means that when we use the aws cli, we have a base profile (that we don't set as default for security reasons) with credentials and then every other profile is an assume role with that base profile as the source profile. So when using this module, it does not work because I have no way of specifying the profile I want the provider to use. Can this be added?

So for example, this is how we set up our providers:

provider "aws" {
  alias           = "other-account"
  region          = "us-east-1"
  profile         = "ops"
  assume_role {
    role_arn      = "arn:aws:iam::XXXXXXXXXXXX:role/peering-assume-role"
  }
}

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Describe the Bug

When setting enabled to false you get errors in the aws_vpc_peering_connection_options.accepter and the aws_vpc_peering_connection_options.requester. The errors seem to be because these two resources are missing count.

Expected Behavior

When I set enabled to false I expect the module to do nothing.

Steps to Reproduce

Steps to reproduce the behavior:

1. Set enabled to false with other variables filled with empty strings
2. Run terraform plan (no errors should be shown)
3. Run terraform apply (see errors)

Error Message

Error: error reading VPC Peering Connection (): InvalidVpcPeeringConnectionId.Malformed: Invalid id: "" (expecting "pcx-...")
        status code: 400, request id: c62fad7b-fc41-45d5-9fcb-6b97d7836638

  on .terraform/modules/vpc_peering_2/accepter.tf line 105, in resource "aws_vpc_peering_connection_options" "accepter":
 105: resource "aws_vpc_peering_connection_options" "accepter" {



Error: error reading VPC Peering Connection (): InvalidVpcPeeringConnectionId.Malformed: Invalid id: "" (expecting "pcx-...")
        status code: 400, request id: fc8948ec-92c9-4901-834d-9e6263055096

  on .terraform/modules/vpc_peering_2/requester.tf line 120, in resource "aws_vpc_peering_connection_options" "requester":
 120: resource "aws_vpc_peering_connection_options" "requester" {

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• OS: Ubuntu 18.04.4
• Code Version: 0.7.0 (but I see the missing count in master)

Additional Context

I'm trying to loop through a list of VPC Peers but due to count in modules not being supported I've created three modules that I've set to enabled=false if the list of VPC Peers doesn't have a second or third element.

locals {
  # A dirty hack to support the below module hack
  vpc_peers = [for index in [0, 1, 2] : {
    peer_vpc_id    = (length(var.vpc_peers) > index) ? data.terraform_remote_state.aws_peer_vpcs[index].outputs.vpc_id : ""
    peer_region    = (length(var.vpc_peers) > index) ? data.terraform_remote_state.aws_peer_vpcs[index].outputs.region : ""
    peer_vpc_name  = (length(var.vpc_peers) > index) ? data.terraform_remote_state.aws_peer_vpcs[index].outputs.vpc_name : ""
    requester_role = (length(var.vpc_peers) > index) ? var.vpc_peers[index].requester_role : ""
    accepter_role  = (length(var.vpc_peers) > index) ? var.vpc_peers[index].accepter_role : ""
    enabled        = length(var.vpc_peers) > index
  }]
}

# A dirty hack as count and for_each is not supported for modules until 0.13 (currently 0.12)
module "vpc_peering_1" {
  source    = "git::https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account.git?ref=0.7.0"
  namespace = terraform.workspace
  stage     = var.env
  name      = "${data.terraform_remote_state.this.outputs.vpc_name}-to-${local.vpc_peers[0].peer_vpc_name}"
  enabled   = local.vpc_peers[0].enabled

  requester_aws_assume_role_arn             = local.vpc_peers[0].requester_role
  requester_region                          = var.aws_region
  requester_vpc_id                          = data.terraform_remote_state.this.outputs.vpc_id
  requester_allow_remote_vpc_dns_resolution = true

  accepter_aws_assume_role_arn             = local.vpc_peers[0].accepter_role
  accepter_region                          = var.aws_region
  accepter_vpc_id                          = local.vpc_peers[0].peer_vpc_id
  accepter_allow_remote_vpc_dns_resolution = true
}

# A dirty hack as count and for_each is not supported for modules until 0.13 (currently 0.12)
module "vpc_peering_2" {
  source    = "git::https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account.git?ref=0.7.0"
  namespace = terraform.workspace
  stage     = var.env
  name      = "${data.terraform_remote_state.this.outputs.vpc_name}-to-${local.vpc_peers[0].peer_vpc_name}"
  enabled   = local.vpc_peers[1].enabled

  requester_aws_assume_role_arn             = local.vpc_peers[1].requester_role
  requester_region                          = var.aws_region
  requester_vpc_id                          = data.terraform_remote_state.this.outputs.vpc_id
  requester_allow_remote_vpc_dns_resolution = true

  accepter_aws_assume_role_arn             = local.vpc_peers[1].accepter_role
  accepter_region                          = var.aws_region
  accepter_vpc_id                          = local.vpc_peers[1].peer_vpc_id
  accepter_allow_remote_vpc_dns_resolution = true
}

Currently the module is inoperable with Terraform 0.12 code, it would be great if it could support Terraform 0.12 :)

Describe the Bug

The module requires version 2.x of the AWS provider, which is no longer under active development.

Expected Behavior

If included in a project that uses version 3.x of the AWS provider, the module should work properly.

Steps to Reproduce

Reference the module from a project with a newer version of the AWS provider, e.g., one that has

terraform {
  required_providers {
    aws = {
      source  = "aws"
      version = "~> 3.10"
    }
  }
}

Running terraform init will print an error because it can't satisfy the version constraints of the project and the module at the same time.

Environment (please complete the following information):

Terraform 0.13.4, AWS provider 3.10.0.

Additional Context

I tried modifying a local copy of the module to relax the version constraint (changed it from ~> 2.0 to >= 2.0) and it appeared to work fine with provider version 3.10.0, though I didn't test every possible permutation of inputs.

When I run terraform plan I get the below error

module.vpc_peering_cross_account: eval: *terraform.EvalLocal, err: Missing resource instance key: Because data.aws_route_tables.accepter has "count" set, its attributes must be accessed on specific instances.

For example, to correlate with indices of a referring resource, use:
data.aws_route_tables.accepter[count.index]

I have a use case where I need to peer vpcs with a third-party company (and therefore cannot assume a role in the accepter account). I'd like to still be able to manage my side of these vpc peering connections in terraform, even if I can't manage the accepter infrastructure. Could cloudposse add a flag to enable/disable accepter resources here?

Note also that accepter resources should be enabled for auto_accept to be true.

Found a bug? Maybe our Slack Community can help.

Describe the Bug

When calculating the local value accepter_aws_rt_map the try condition always fails to default because it's attempting to access a set value using an index which is not possible.

Expected Behavior

The try condition properly accesses the set values when available to correctly build the map.

Steps to Reproduce

Steps to reproduce the behavior:

1. Create a VPC with multiple route tables attached to different subnets as the accepter
2. Run the terraform module with accepted and requester
3. See that only one route table is mapped to the peering connection

Screenshots

If applicable, add screenshots or logs to help explain your problem.

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• Terraform Version 0.13
• AWS Provider Version 4.0

Additional Context

Wrapping the set value in "tolist()" should resolve this, but I have yet to fully validate this solution.

Hi,

I am having problems with the module to assume the roles:

Error: Error refreshing state: 2 error(s) occurred:

* module.vpc_peering_cross_account.provider.aws.accepter: The role "arn:aws:iam::YYYYYYYYYYYYY:role/cross_account_role" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid
* module.vpc_peering_cross_account.provider.aws.requester: The role "arn:aws:iam::XXXXXXXXXXX:role/cross_account_role" cannot be assumed.

  There are a number of possible causes of this - the most common are:
    * The credentials used in order to assume the role are invalid
    * The credentials do not have appropriate permission to assume the role
    * The role ARN is not valid

Using the same credentials and roles I can assume the roles using aws cli.

Any idea what can cause it?

Thanks

Describe the Bug

We are getting the following error when running apply. We are in the process of upgrading from Terraform 0.11 to Terraform 0.12. Previously this was successful on Terraform 0.11 using the 0.11 branch.

Error: Invalid index

  on .terraform/modules/staging_peering/accepter.tf line 88, in resource "aws_route" "accepter":
  88:   route_table_id            = local.accepter_aws_route_table_ids[ceil(count.index / local.requester_cidr_block_associations_count)]
    |----------------
    | count.index is 5
    | local.accepter_aws_route_table_ids is list of string with 3 elements
    | local.requester_cidr_block_associations_count is 2

The given key does not identify an element in this collection value.

Expected Behavior

Plan runs without error.

Steps to Reproduce

Steps to reproduce the behavior:

1. Configure the module as follows.

module "staging_peering" {
  source  = "cloudposse/vpc-peering-multi-account/aws"
  version = "0.11.3"

  namespace = "staging"
  stage     = "staging"
  name      = "staging-shared-services-peering"

  requester_aws_assume_role_arn             = "arn:aws:iam::xxxxxx:role/cross-account-vpc-peering"
  requester_region                          = "eu-west-1"
  requester_vpc_id                          = data.aws_vpc.peer_from_vpc.id
  requester_allow_remote_vpc_dns_resolution = false

  accepter_aws_assume_role_arn             = "arn:aws:iam::xxxxxx:role/pipeline"
  accepter_region                          = "us-east-2"
  accepter_vpc_id                          = data.aws_vpc.peer_to_staging_vpc.id
  accepter_allow_remote_vpc_dns_resolution = false
}

2. Run 'terraform plan.'
3. See error

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

• Version 0.11.3
• Terraform 0.12.30

Describe the Bug

accepter_subnet_tags doesn't have any effect, data.aws_route_tables.accepter always contains the all route tables in accepter VPS (link to code)

requester_subnet_tags works as expected.

Expected Behavior

Just works as described in the readme.

Steps to Reproduce

Code to reproduce the behavior:

module "prod_to_stage" {
  source = "git::https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account.git?ref=tags/0.11.0"

  namespace = "namespace"
  stage     = "stage"
  name      = "prod-to-stage"

  requester_aws_assume_role_arn             = "arn:aws:iam::${var.account_id}:role/cross-account-vpc-peering-test"
  requester_region                          = var.region
  requester_vpc_id                          = var.prod_vpc_id
  requester_allow_remote_vpc_dns_resolution = true
  requester_subnet_tags                     = {
    Role = "subnet-private"
  }

  accepter_aws_assume_role_arn             = "arn:aws:iam::${var.stage_account_id}:role/cross-account-vpc-peering-test"
  accepter_region                          = var.region
  accepter_vpc_id                          = var.stage_vpc_id
  accepter_allow_remote_vpc_dns_resolution = true
  accepter_subnet_tags                     = {
    "Role" = "subnet-private"
  }
}

Environment

Anything that will help us triage the bug will help. Here are some ideas:

• terraform 0.12.29
• terraform-aws-vpc-peering-multi-account 0.11.0

Error: Error applying plan:

2 error(s) occurred:

* module.vpc_peering_syn_scout_dev.aws_vpc_peering_connection_options.requester: 1 error(s) occurred:

* aws_vpc_peering_connection_options.requester: Error modifying VPC Peering Connection Options: OperationNotPermitted: Peering pcx-0324fc216d5832fd3 is not active. Peering options can be added only to active peerings.
	status code: 400, request id: a0a31b88-9f31-4f02-a37f-5863bf16ec36
* module.vpc_peering_syn_scout_dev.aws_vpc_peering_connection_options.accepter: 1 error(s) occurred:

* aws_vpc_peering_connection_options.accepter: Error modifying VPC Peering Connection Options: OperationNotPermitted: Peering pcx-0324fc216d5832fd3 is not active. Peering options can be added only to active peerings.
	status code: 400, request id: f4fdc3cc-9a85-4d44-827b-2faa56b5a0ae

It seems as though the aws_vpc_peering_connection_options resource will not work until the peering connections are active. Is there any way for the aws_vpc_peering_connection_options resources to wait for the peers to be active before running them?

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Describe the Feature

Instead of using a hard coded requester_vpc_id, we can use a data source instead.

data "aws_vpc" "selected" {
  provider = aws.accepter

  tags = var.accepter_vpc_tags
}

where the tags are passed in

  accepter_vpc_tags = {
    Name = "yellowstone"
  }

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update terraform cloudposse/vpc/aws to v2.2.0

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update terraform cloudposse/dynamic-subnets/aws to v2.4.2

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository