Giter Club home page Giter Club logo

terraform-aws-base-networking's Introduction

Terraform AWS Base Networking

Version Build Pipeline Maintainer

A Terraform module for building a base network in AWS.

The network consists of:

  • Public and private subnets for each supplied availability zone
  • A NAT gateway for each supplied availability zone for outbound Internet connectivity
  • Routes from the public subnets to the Internet gateway
  • Routes from the private subnets to the NAT
  • Standard tags for all resources

Diagram of infrastructure managed by this module

Usage

To use the module, include something like the following in your Terraform configuration:

module "base-network" {
  source  = "infrablocks/base-networking/aws"
  version = "4.0.0"

  vpc_cidr           = "10.0.0.0/16"
  region             = "eu-west-2"
  availability_zones = ["eu-west-2a", "eu-west-2b"]

  component             = "important-component"
  deployment_identifier = "production"

  private_zone_id = "Z3CVA9QD5NHSW3"
}

See the Terraform registry entry for more details.

Inputs

Name Description Default Required
vpc_cidr The CIDR to use for the VPC. - Yes
region The region into which to deploy the VPC. - Yes
availability_zones The availability zones for which to add subnets. - Yes
public_subnets_offset The number of /24s to offset the public subnets in the VPC CIDR. 0 No
private_subnets_offset The number of /24s to offset the private subnets in the VPC CIDR. 0 No
component The component this network will contain. - Yes
deployment_identifier An identifier for this instantiation. - Yes
dependencies A comma separated list of components depended on my this component. [] No
include_route53_zone_association Whether or not to associate VPC with the private Route 53 zone ("yes" or "no"). "yes" No
private_zone_id The ID of the private Route 53 zone` - If include_route53_zone_association is "yes"
include_nat_gateways Whether or not to deploy NAT gateways for outbound Internet connectivity ("yes" or "no"). "yes" No

Outputs

Name Description
vpc_id The ID of the created VPC.
vpc_cidr The CIDR of the created VPC.
availability_zones The availability zones in which subnets were created.
number_of_availability_zones The number of populated availability zones available.
public_subnet_ids The IDs of the public subnets.
public_subnet_cidrs The CIDRs of the public subnets.
public_route_table_ids The IDs of the public route tables.
private_subnet_ids The IDs of the private subnets.
private_subnet_cidrs The CIDRs of the private subnets.
private_route_table_ids The IDs of the private route tables.
nat_public_ips The EIPs attached to the NAT gateways.
internet_gateway_id The ID of the created IGW.

Compatibility

This module is compatible with Terraform versions greater than or equal to Terraform 1.0 and Terraform AWS provider versions greater than or equal to 3.27.

Required Permissions

  • ec2:DescribeVpcs
  • ec2:DescribeAddresses
  • ec2:DescribeVpcAttribute
  • ec2:DescribeVpcClassicLink
  • ec2:DescribeVpcClassicLinkDnsSupport
  • ec2:DescribeRouteTables
  • ec2:DescribeSecurityGroups
  • ec2:DescribeNetworkAcls
  • ec2:DescribeSubnets
  • ec2:DescribeInternetGateways
  • ec2:DescribeNatGateways
  • ec2:ModifyVpcAttribute
  • ec2:AllocateAddress
  • ec2:ReleaseAddress
  • ec2:AssociateRouteTable
  • ec2:DisassociateRouteTable
  • ec2:AttachInternetGateway
  • ec2:DetachInternetGateway
  • ec2:DeleteInternetGateway
  • ec2:CreateRoute
  • ec2:CreateNatGateway
  • ec2:CreateVpc
  • ec2:CreateTags
  • ec2:CreateSubnet
  • ec2:CreateRouteTable
  • ec2:CreateInternetGateway
  • ec2:DeleteRoute
  • ec2:DeleteRouteTable
  • ec2:DeleteSubnet
  • ec2:DeleteNatGateway
  • ec2:DeleteVpc
  • s3:ListBucket
  • s3:GetObject
  • s3:GetObjectTagging
  • s3:DeleteObject
  • route53:AssociateVPCWithHostedZone
  • route53:DisassociateVPCFromHostedZone
  • route53:GetChange
  • route53:GetHostedZone

Development

Machine Requirements

In order for the build to run correctly, a few tools will need to be installed on your development machine:

  • Ruby (3.1)
  • Bundler
  • git
  • git-crypt
  • gnupg
  • direnv
  • aws-vault

Mac OS X Setup

Installing the required tools is best managed by homebrew.

To install homebrew:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Then, to install the required tools:

# ruby
brew install rbenv
brew install ruby-build
echo 'eval "$(rbenv init - bash)"' >> ~/.bash_profile
echo 'eval "$(rbenv init - zsh)"' >> ~/.zshrc
eval "$(rbenv init -)"
rbenv install 3.1.1
rbenv rehash
rbenv local 3.1.1
gem install bundler

# git, git-crypt, gnupg
brew install git
brew install git-crypt
brew install gnupg

# aws-vault
brew cask install

# direnv
brew install direnv
echo "$(direnv hook bash)" >> ~/.bash_profile
echo "$(direnv hook zsh)" >> ~/.zshrc
eval "$(direnv hook $SHELL)"

direnv allow <repository-directory>

Running the build

Running the build requires an AWS account and AWS credentials. You are free to configure credentials however you like as long as an access key ID and secret access key are available. These instructions utilise aws-vault which makes credential management easy and secure.

To run the full build, including unit and integration tests, execute:

aws-vault exec <profile> -- ./go

To run the unit tests, execute:

aws-vault exec <profile> -- ./go test:unit

To run the integration tests, execute:

aws-vault exec <profile> -- ./go test:integration

To provision the module prerequisites:

aws-vault exec <profile> -- ./go deployment:prerequisites:provision[<deployment_identifier>]

To provision the module contents:

aws-vault exec <profile> -- ./go deployment:root:provision[<deployment_identifier>]

To destroy the module contents:

aws-vault exec <profile> -- ./go deployment:root:destroy[<deployment_identifier>]

To destroy the module prerequisites:

aws-vault exec <profile> -- ./go deployment:prerequisites:destroy[<deployment_identifier>]

Configuration parameters can be overridden via environment variables. For example, to run the unit tests with a seed of "testing", execute:

SEED=testing aws-vault exec <profile> -- ./go test:unit

When a seed is provided via an environment variable, infrastructure will not be destroyed at the end of test execution. This can be useful during development to avoid lengthy provision and destroy cycles.

To subsequently destroy unit test infrastructure for a given seed:

FORCE_DESTROY=yes SEED=testing aws-vault exec <profile> -- ./go test:unit

Common Tasks

Generating an SSH key pair

To generate an SSH key pair:

ssh-keygen -m PEM -t rsa -b 4096 -C [email protected] -N '' -f config/secrets/keys/bastion/ssh

Generating a self-signed certificate

To generate a self signed certificate:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365

To decrypt the resulting key:

openssl rsa -in key.pem -out ssl.key

Managing CircleCI keys

To encrypt a GPG key for use by CircleCI:

openssl aes-256-cbc \
  -e \
  -md sha1 \
  -in ./config/secrets/ci/gpg.private \
  -out ./.circleci/gpg.private.enc \
  -k "<passphrase>"

To check decryption is working correctly:

openssl aes-256-cbc \
  -d \
  -md sha1 \
  -in ./.circleci/gpg.private.enc \
  -k "<passphrase>"

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/infrablocks/terraform-aws-base-networking. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The library is available as open source under the terms of the MIT License.

terraform-aws-base-networking's People

Contributors

bamdadd avatar dependabot[bot] avatar gryff avatar infrablocks-maintainers avatar jonassvalin avatar tobyclemson avatar vlad-ro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-aws-base-networking's Issues

Conflict with private zone association

I've got a project using terraform-aws-dns-zones as well as terraform-aws-base-networking, and I'm getting this error:

* module.base-network.aws_route53_zone_association.base: 1 error(s) occurred:

* aws_route53_zone_association.base: ConflictingDomainExists: The VPC that you chose, vpc-6705bd01 in region eu-west-1, is already associated with another private hosted zone that has an overlapping name space, default.private.cloudspin.net..
	status code: 400, request id: e156c4f4-2843-11e8-bfa5-7966af78f306

I believe this is because the association between the dns zone and the vpc is effectively being declared twice: once when the zone is created in terraform-aws-dns-zones , and then again with a zone association statement in terraform-aws-base-networking

My code to use the dns module (0.1.7) is:

module "dns-zones" {
  source = "infrablocks/dns-zones/aws"
  version = "~> 0.1"

  domain_name             = "${var.public_domain_name}"
  private_domain_name     = "${var.private_domain_name}"

  private_zone_vpc_region = "${var.region}"
  private_zone_vpc_id     = "${module.base-network.vpc_id}"
}

My code to use the networking module (0.1.24) is:

module "base-network" {
  source  = "infrablocks/base-networking/aws"
  version = "~> 0.1"
  
  vpc_cidr = "10.1.0.0/16"
  region = "${var.region}"

  availability_zones = "${var.availability_zones}"
  
  component = "${var.component_base}-${var.env_name}"
  deployment_identifier = "${var.env_name}"
  include_lifecycle_events = "no"
  
  private_zone_id = "${module.dns-zones.private_zone_id}"
}

Looking at the source for the modules, in the dns module I see:

resource "aws_route53_zone" "private_zone" {
  name = "${var.private_domain_name}"
  vpc_id = "${var.private_zone_vpc_id}"
  vpc_region = "${var.private_zone_vpc_region}"
}

and in the networking module I see:

resource "aws_route53_zone_association" "base" {
  zone_id = "${var.private_zone_id}"
  vpc_id = "${aws_vpc.base.id}"
}

These look like they're trying to set the same thing.

The terraform docs imply that aws_route53_zone_association is not meant to declare the same vpc-id:

resource "aws_route53_zone" "example" {
  name   = "example.com"
  vpc_id = "${aws_vpc.primary.id}"
}

resource "aws_route53_zone_association" "secondary" {
  zone_id = "${aws_route53_zone.example.zone_id}"
  vpc_id  = "${aws_vpc.secondary.id}"
}

How are you using this? I don't see a workaround unless I create the private zone for a different vpc?

Problem using the module twice in a project

I'm trying to create two instances of the base networking in a single Terraform project. To be fair, I'm doing this to demonstrate why this is a Bad Idea. Nevertheless, it probably ought to work.

I'm getting an error about the provider; this might suggest that the module should not define a provider, as per the tf module docs:

While in principle provider blocks can appear in any module, it is recommended that they be placed only in the root module of a configuration

My project has two module declarations, one like this:

module "prod-base-network" {
  source  = "infrablocks/base-networking/aws"
  ...

The other being named stage-base-network.

The error I get when I run terraform plan is:

Error: Error asking for user input: 1 error(s) occurred:

* module.base-network.aws_nat_gateway.base: configuration for module.base-network.provider.aws is not present; a provider configuration block is required for all operations

The tf documentation above sounds like it might be best to remove the provider.tf from the module source. I suspect that will have knock-on effects on building and testing the module, which I've so far avoided diving into.

Example adding instances from EC2?

Hi! ๐Ÿ‘‹ I'm new to terraform, and I'm looking to create a set of EC2 instances that can see one another. I've done this on Google Cloud, and for that strategy I use a common module to setup networking, and then I have some logic in the startup script to use the Google metadata API to get the ips for other instances. I'm looking for a similar setup for AWS. To be specific, my questions are:

  • Is there an example using this module alongside creating instances that can easily see one another?
  • What are best practices to share ip addresses between the instances?

Thanks for your help!

include_nat_gateway as a variable name in 2.3.0

Hello

I think module version 2.3.0 uses include_nat_gateway as a variable name while readme and TF registry mention include_nat_gateways.

[steph@centos7-2 MyFirstVPC]$ ti -upgrade
Upgrading modules...
Downloading infrablocks/base-networking/aws 2.3.0 for base-network...

  • base-network in .terraform/modules/base-network

Initializing the backend...

Initializing provider plugins...

  • Finding hashicorp/aws versions matching "~> 3.0"...
  • Installing hashicorp/aws v3.23.0...
  • Installed hashicorp/aws v3.23.0 (signed by HashiCorp)

[steph@centos7-2 MyFirstVPC]$ tv

Error: Unsupported argument

on vpc.tf line 30, in module "base-network":
30: include_nat_gateways = "no"

An argument named "include_nat_gateways" is not expected here. Did you mean
"include_nat_gateway"?

[steph@centos7-2 MyFirstVPC]$ grep -r "include_nat_gateway" .
./.terraform/modules/base-network/config/roles/harness.yaml: include_nat_gateway: "%{hiera('include_nat_gateway')}"
./.terraform/modules/base-network/config/defaults.yaml:include_nat_gateway: "yes"
./.terraform/modules/base-network/spec/infra/harness/main.tf: include_nat_gateway = var.include_nat_gateway
./.terraform/modules/base-network/spec/infra/harness/variables.tf:variable "include_nat_gateway" {}
./.terraform/modules/base-network/spec/nat_spec.rb: context 'when include_nat_gateway is no' do
./.terraform/modules/base-network/spec/nat_spec.rb: reprovision(include_nat_gateway: "no")
./.terraform/modules/base-network/spec/nat_spec.rb: context 'when include_nat_gateway is yes' do
./.terraform/modules/base-network/spec/nat_spec.rb: reprovision(include_nat_gateway: "yes")
./.terraform/modules/base-network/README.md:| include_nat_gateway | Whether or not to deploy a NAT gateway for outbound Internet connectivity | true | yes |
./.terraform/modules/base-network/nat.tf: count = var.include_nat_gateway == "yes" ? 1 : 0
./.terraform/modules/base-network/nat.tf: count = var.include_nat_gateway == "yes" ? 1 : 0
./.terraform/modules/base-network/private_subnets.tf: count = var.include_nat_gateway == "yes" ? 1 : 0
./.terraform/modules/base-network/variables.tf:variable "include_nat_gateway" {
./vpc.tf: include_nat_gateway = "no"
[steph@centos7-2 MyFirstVPC]$ grep -r "include_nat_gateways" .
[steph@centos7-2 MyFirstVPC]$

Adding availability zones causes subnet conflicts

For a given network only using 1 or 2 availability zones, adding further availability zones causes conflicts with the existing subnets, as the subnets are likely indexed by order in collection. This can cause Terraform to think it needs to destroy the existing ones and recreate everything, making it impossible to add availability zones.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.