Giter Club home page Giter Club logo

startup-kit-templates's Introduction

Overview

The StartupKit-templates repo contains a collection of AWS CloudFormation templates intended to help you set up common pieces of AWS infrastructure. Each template defines a stack, which is a collection of related resources that can be created, updated, or deleted as a single unit. Templates are available for creating:

The VPC template is a requirement for the others. You can either run the templates/vpc.cfn.yml template by itself prior to using the others, or run any one of the vpc-*.cfn.yml wrapper templates at the top level of this repo to create sets of resources. For example, vpc-bastion-fargate-rds.cfn.yml will create a single stack containing a vpc, bastion host, fargate cluster, and database.

StartupKit is designed to be modular. Some stacks depend on others, some can be deployed individually or in combination with others. You can use the stacks for each module individually and combine them on your own, or use wrapper stacks we have created from the tables below that provide one-click launch for common combinations. The wrapper stacks in the one-click launch table are broken down by regions in order to simplify deployments. See the Region Table for more information on availability of services by region.

Prerequisites

If you haven't already done so you first need to:

Creating stacks

Use the AWS CloudFormation Console to run the templates. Click the "Create Stack" button in the upper left corner of the console, then under "Choose a template", select "Upload a template to Amazon S3" and click "Browse" to find your local fork of this repository and choose the template you want to run.

To launch stacks directly directly from this README see the table below.

The templates

Each section contains details about template parameters and the resources created by the stack.

VPC

The vpc.cfn.yml template is a prerequisite for most of the others--you need to either run it first, or run one of the wrapper templates at the top level of the repo, which include it. It creates a private networking environment in which you can securely run AWS resources, along with related networking resources.

Subnets are isolated network areas--resources in public subnets are visible to the Internet, resources in private subnets can only be reached from inside the VPC. If a resource in a private subnet needs to communicate externally it has to do so via a NAT Gateway, which acts as a proxy.

The VPC template creates two public and two private subnets, in different Availability Zones (AZ) for redundancy. A subnet is public if it’s associated with an Internet gateway, which allow it to communicate with the Internet

Each subnet has to be associated with a route table, or set of network rules, that define allowed traffic. Route tables operate at the subnet level. The VPC template creates two of them: one for the public subnets, and one for the private.

Security groups act as firewalls at the instance level, to control inbound and outbound traffic. The template creates security groups for an application, load balancer, database, and bastion host. Depending on what other templates you run, not all of them may be used.

Resources Created
Diagram

VPC

Bastion Host

It is preferable not to ssh into EC2 instances at all, instead monitoring instances by configuring them to send logs to CloudWatch or other services, and managing instantiation, configuration, and termination of instances using devops tools.

If you do need to connect directly to instances, it's best (and for instances in a private subnets, a requirement) to use a bastion host, otherwise known as a jump box. A bastion host is an EC2 instance that is publicly accessible, and also has access to private resources, allowing it to function as a secure go-between. You configure your EC2 instances to only accept ssh traffic from the bastion host, then you can ssh into the bastion host, and from there connect to your private resources.

EC2 key pairs are required to ssh into any EC2 instance, including bastion hosts. If an attacker gains access to your key pair, they can use it to get into your bastion host, and thus your other resources. In order to prevent this kind of breach the bastion host template supports enabling Multi-Factor Authentication (MFA), which is highly recommended

With MFA enabled you use an app like Google Authenticator or Authy to obtain a one-time password, and use this when logging in, in addition to your username and key pair.

You can also set how long CloudWatch logs are retained, and optionally enable Multi-Factor Authentication, among other options.

Creating a Bastion Host stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

After the bastion stack has been created, you can log into the EC2 section of the console, find the EC2 instance containing the stack name, copy its public DNS address, and ssh into it. Once on the bastion host you should be able to reach all AWS resources running in the same VPC.

For security and cost optimization it is a best practice to stop (not terminate!) the bastion host when not in use.

See Enabling Multi-factor authentication on the Bastion Host for additional MFA information.

Resources Created
Diagram

VPC + Bastion Host

AWS Elastic Beanstalk

AWS Elastic Beanstalk is a service that lets you define an environment for common application types, and deploy code into it. The Beanstalk template is dependent on the VPC, and optionally can be used with the bastion, RDS, or Aurora templates.

Creating a AWS Elastic Beanstalk stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

The elastic-beanstalk.cfn.yml template asks for a series of inputs defining your environment. Those with constrained values are:

  • A stack type, with allowed values of node, rails, python, python3 or spring.
  • An environment name with allowed values of dev or prod.
  • The name of the stack you previously created to define your VPC, as the NetworkStackName parameter.
Resources Created
Diagram

VPC + Bastion + Elastic Beanstalk + DB

AWS Fargate

AWS Fargate is part of Amazon Elastic Container Service (ECS). It's a managed service for running container-based applications, without having to worry about the underlying servers--sort of like Lambda for containers.

Creating a Fargate stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

Resources Created
Diagrams

With RDS/Aurora: VPC + Bastion + Fargate + DB

Without RDS/Aurora:

VPC + Bastion + Fargate

Amazon RDS

Amazon Relational Database Service (RDS) is a service for running relational databases without having to manage the server software, backups, or other maintenance tasks. The RDS service as a whole supports Amazon Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server; this template currently works with PostgreSQL, MySQL, and MariaDB, and supports t2, m4, and r4 instance types.

Creating an RDS stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

Resources Created
  • A DB instance
  • A DB subnet group

Amazon Aurora

Amazon Aurora is a high-performance cloud-optimized relational database, which is compatible with MySQL and PostgreSQL. It’s treated separately than RDS because Aurora has a few unique characteristics.

Creating an Aurora stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

Resources Created

Amazon ElastiCache Cluster

Amazon ElastiCache is a managed high-performance in-memory data store, backed with either the Redis or Memcached engines. Running this template lets you select the engine type, number of nodes in the cluser, and the instance type of the nodes.

Creating an ElastiCache stack requires you to have first created a VPC stack, and to enter the name of the VPC stack as the NetworkStackName parameter.

Resources Created

Billing Alerts

If you leave AWS resources running longer than intended, have unexpected traffic levels, or misconfigure or over provision resources, your bill can climb higher or faster than expected. To avoid surprises we recommend turning on billing alerts, so that you're notified when charges go above preconfigured thresholds. The billing alert template makes this easier.

Before running you need to use the AWS console to enable billing alerts:

  • Log into the billing section of the console. Click your username on the top right and select 'My Billing Dashboard.'
  • Select 'Preferences' from the list of options on the left.
  • Check 'Receive Billing Alerts.' Once saved this cannot be disabled.

Now you can run the billing_alert.cfn.yml template, which will create a CloudWatch alarm and an SNS topic. You'll be asked for the threshold (in US dollars) for receiving an alert and the email address the alert should be sent to. If you want to get alerts at more than one threshold, you can run the template multiple times.

You can read about more ways to avoid unexpected charges.

Launching Modular Stacks

Select the Category of stack you want to launch below. Then find the row with the combination of modules you are looking for from the checkbox columns (i.e. vpc+bastion host) and select the region you want to launch the stack in. Click 'Launch Stack' button and the CloudFormation console will open automatically with the stack's details.

New services are not immediately available in all AWS Regions, please consult the Region Table for more information.

Basic Infrastructure Templates (VPC etc)
CloudFormation Region Name Region VPC Bastion
US East (N. Virginia) us-east-1
US East (N. Virginia) us-east-1
US East (Ohio) us-east-2
US East (Ohio) us-east-2
US West (N. California) us-west-1
US West (N. California) us-west-1
Canada (Central) ca-central-1
Canada (Central) ca-central-1
S. America (São Paulo) sa-east-1
S. America (São Paulo) sa-east-1
EU (Ireland) eu-west-1
EU (Ireland) eu-west-1
EU (London) eu-west-2
EU (London) eu-west-2
EU (Paris) eu-west-3
EU (Paris) eu-west-3
EU (Frankfurt) eu-central-1
EU (Frankfurt) eu-central-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Sydney) ap-southeast-2
AWS Elastic Beanstalk
CloudFormation Region Name Region VPC Bastion DB Elastic Beanstalk
US East (N. Virginia) us-east-1
US East (Ohio) us-east-2
US West (N. California) us-west-1
US West (Oregon) us-west-2
Canada (Central) ca-central-1
S. America (São Paulo) sa-east-1
EU (Ireland) eu-west-1
EU (London) eu-west-2
EU (Paris) eu-west-3
EU (Frankfurt) eu-central-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
AWS Fargate
CloudFormation Region Name Region VPC Bastion DB Fargate
US East (N. Virginia) us-east-1
US East (N. Virginia) us-east-1
US East (Ohio) us-east-2
US East (Ohio) us-east-2
US West (Oregon) us-west-2
US West (Oregon) us-west-2
EU (Ireland) eu-west-1
EU (Ireland) eu-west-1
EU (Frankfurt) eu-central-1
EU (Frankfurt) eu-central-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Tokyo) ap-northeast-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Sydney) ap-southeast-2

startup-kit-templates's People

Contributors

bdiesel avatar dgeske avatar hyandell avatar john avatar koroshan avatar laardee avatar mikeapted avatar paulfryer avatar prafulmehrotra avatar rabowskyb avatar rcowper avatar rnzsgh avatar sotayamashita avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

startup-kit-templates's Issues

Incorrect security group ingress and egress

The use of port ranges in three of the egress/ingress rules looks wrong in the VPC definition:

`Existing

BastionSecurityGroupToDbEgress:
    Type: AWS::EC2::SecurityGroupEgress  # prevent security group circular references
    Properties:
      GroupId: !Ref BastionSecurityGroup
      IpProtocol: tcp
      ToPort: 5432
      FromPort: 3306
      DestinationSecurityGroupId: !Ref DbSecurityGroup

`

That's opening thousands of ports! Perhaps it would be better to split it into two:

`New

BastionSecurityGroupToDbEgress:
    Type: AWS::EC2::SecurityGroupEgress  # prevent security group circular references
    Properties:
      GroupId: !Ref BastionSecurityGroup
      IpProtocol: tcp
      ToPort: 3306
      FromPort: 3306
      DestinationSecurityGroupId: !Ref DbSecurityGroup

BastionSecurityGroupToDbEgress:
    Type: AWS::EC2::SecurityGroupEgress  # prevent security group circular references
    Properties:
      GroupId: !Ref BastionSecurityGroup
      IpProtocol: tcp
      ToPort: 5432
      FromPort: 5432
      DestinationSecurityGroupId: !Ref DbSecurityGroup

`

Please add DocumentDB example

It would be great to have a documentDB example importing VPC resources. I have tried adjusting the db template, adjusting from RDS to DocDB, but regardless of how I specify "VPCSecurityGroupIds" (tried with import, just like the RDS example, tried merging the VPC and DocDB template and using ref), but DocDB database creation always fails with Property validation failure: [Encountered unsupported properties in {/}: [VPCSecurityGroupIds]].

Find attached the example VPC-cloudformation and the DocumentDB cloudformation.

test-documentdb.zip

Test framework ( TaskCat ) is a must to comply with SDLC

Since the infrastructure becomes a code (software), any code should be testable according to SDLC.

Then, we need to automate testing on the CloudFormation templates used in this repository.

Currently, I can suggest the following:

1. Syntax Validity

aws cloudformation validate-template is designed to check the syntax of the template, but it does not check that the values being passed on to properties are valid.

2. Operational Validity

TaskCat automates the testing of templates in multiple regions because, currently, the only method to check the operational validity of a template is to attempt to create a stack.. This is the same tool used internally by AWS teams to test the AWS Quick Start templates.

Whenever I have time, I should contribute in this great repository.

Bastion issue

I'm getting some issues when trying to use the bastion.cfn.yml cloud formation template:

CREATE FAILED
"You must specify an allocation id when mapping an address to a VPC instance"

ROLLBACK_IN_PROGRESS
Next I got "Rollback in progress" alert and that message:
The following resource(s) failed to create: [BastionEIP]. . Rollback requested by user.

Am I doing something wrong? Or there is a bug on bastion template?

Template format error: Output Name is malformed.

running aws validate-template on the template vpn-cfn.yml gives an error concerning the template outputs

An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Output Name is malformed. The Name field of every Export member must be specified and consist only of alphanumeric characters, colons, or hyphens.

Fargate stack not adding the application security group to ALB

I used the combination of vpc+bastion stack, a separate db stack (postgres) and the fargate stack to run a containerized web app. I found that the load balancer could not connect to the application.

Adding the application security group to the loadbalancer solved this issue.

Organize more the templates according to the software approach DDD

Anything related to network and templates/vpc.fcn.yml can be combined in a single template named network.yml.

The network template should include:

  • Parameters as: health check Ports, Bashion SSH Port (Best practice to not use 22), DB Ports.
  • Resources as:
    • VPC resources: Routers, Subnets,... (which are now provisioned within templates/vpc.fcn.yml)
    • AWS::RDS::DBSubnetGroup: which now provisioned within templates/db.cfn.yml.
    • LoadBalancer: namely if it is a shared ALB among multiple target groups.
  • Outputs: So other templates can refer to the network resources seamlessly (ImportValue+ Export or nested stacks).

I would suggest the same for monitoring, All SNS topics should be in the monitoring template.

reconcile bastion host template with aws-quickstart/quickstart-linux-bastion

A CloudFormation template to create a bastion host has been previously published at aws-quickstart/quickstart-linux-bastion.

The Startup Kit template in this project and the older Quick Start template offer different implementations of the same type of solution, leading to some confusion as to which of the two solutions is the approach recommended by AWS, or if both are still current, what are the strengths/drawbacks or recommended use-cases for each.

Would it be possible to reconcile the two bastion-host templates, whether this involves deprecating one in favor of the other, or merging the features of both into a single unified and well-maintained solution?

I'm wondering if a direct comparison can be made between them, and which approach is more secure, maintainable and/or battle-tested than the other.

(Note that I've opened a parallel issue in the other project at aws-quickstart/quickstart-linux-bastion#36, and I reported a previous duplicate CloudFormation template in aws-quickstart/quickstart-linux-bastion#1 - so this is the third distinct bastion-host CloudFormation template AWS has published that I'm aware of.)

Error when deploy stack vpc-bastion-eb-rds

I deploy stack vpc-bastion-eb-rds by click to on https://github.com/aws-samples/startup-kit-templates (region: ap-southeast-1)
I received an error CREATE_FAILED:

Embedded stack arn:aws:cloudformation:ap-southeast-1:628125106462:stack/xyz-VpcStack-1QVOYUZHS7PKW/f9791870-96b5-11e8-ac8b-500c336f38ba was not successfully created: 
The following resource(s) failed to create: 
[PublicRoute, AppSecurityGroupFromBastionIngress, AppSecurityGroupFromELBIngress, BastionSecurityGroupToAppEgress, DbSecurityGroupFromAppIngress, PublicSubnetRouteTableAssociation1, ELBSecurityGroupToAppEgress, NatEIP2, NatEIP1].

Error 1
Error 2
Error 3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.