aws-samples / codepipeline-nested-cfn Goto Github PK

CloudFormation templates, CodeBuild build specification & Python scripts to perform unit tests of a nested CloudFormation template.

License: MIT No Attribution

Python 100.00%

codepipeline-nested-cfn's Introduction

CodePipeline-Nested-CFN

This repo contains the CloudFormation template which will create a CodePipeline containing multiple stages starting from CodeCommit as source stage, followed by build using CodeBuild, launch test stack, test using CodeBuild, proceed with UAT deployment and wait for manual approval. Once approved it proceed to production stage where it creates a CloudFormation ChangeSet for production stack and wait for approval, once approved it will execute the ChangeSet in production stack.

Let's start by creating the repositories and enabling Continuous Delivery pipeline for nested CFN.

Step 1:

Create base VPC Stack

In the cfn-nested-repo directory there are multiple YAML (CloudFormation Templates) & JSON (CloudFormation Configuration) files.

vpc-stack.yml: is the CloudFormation template to create the base VPC, Subnets, NAT Gateways, etc which will be used. vpc-params.json: is the parameters file which contains the parameter values for the CFN template. Update the ProdApprovalEmail & UATApprovalEmail values to provide the appropriate email address.

Go to cfn-nested-repo directory and execute the following AWS CLI command to create CloudFormation stack.

cd cfn-nested-repo
aws cloudformation create-stack --stack-name NestedCFN-BaseStack --template-body file://vpc-stack.yml --parameters file://vpc-params.json

Step 2:

Update CloudFormation parameters configuration files

In the cfn-nested-repo directory there are following 3 JSON (CloudFormation Configuration) files.

config-test.json: - CloudFormation parameter configuration file for test stack config-uat.json: - CloudFormation parameter configuration file for UAT stack config-prod.json: - CloudFormation parameter configuration file for Prod stack

Update these 3 configuration files with appropriate values for VPCID, PrivateSubnet1, PrivateSubnet2, PublicSubnet1, PublicSubnet2, S3BucketName & DBSubnetGroup based on the values in the output section of the base VPC stack created in Step 1. Update KeyPair value with an existing key pair or create a new key pair and use it.

Step 3:

Creating CodeCommit repositories

Create two CodeCommit repositories as mentioned below.

aws codecommit create-repository --repository-name cfn-nested-repo --repository-description "Repository for CloudFormation templates"

aws codecommit create-repository --repository-name validate-resources --repository-description "Repository for unit testing CloudFormation resources"

Once the repositories are create, clone those repositories and upload the content of directories cfn-nested-repo & validate-resources in their corresponding repositories.

Step 4:

Creating CodePipeline using CloudFormation

Update the codepipeline-cfn-codebuild.json file with the appropriate values for ArtifactStoreS3Location, UATTopic & ProdTopic based on the values from output section of main stack created in Step 1 and update the values for CFNTemplateRepoName & ValidateResourcesRepoName with appropriate values based on the repositories created in Step 3.

Once the configuration file has been updated, execute the following command to create the CloudFormation stack which will create the required CodePipeline.

aws cloudformation create-stack --stack-name NestedCFN-CodePipeline --template-body file://codepipeline-cfn-codebuild.yml --parameters file://codepipeline-cfn-codebuild.json --capabilities CAPABILITY_NAMED_IAM

Once the CloudFormation successfully creates the stack, it would have created a CodePipeline with similar stages as shown below.

Note: While removing the resources, delete the Prod & UAT stacks created by pipeline before deleting the pipeline since those CloudFormations stacks uses the role created by pipeline stack.

codepipeline-nested-cfn's People

Contributors

Stargazers

Watchers

Forkers

p-prakash jevans3 bionicles kleopatra999 jameszh venkatdoddi optionalg unscrambled gridl congdang-agilityio walidmoussa slavah rayl15 sirvict0r jpsriram afaleti mlhatfield amit04pandey mvramana igor-pomogai azsu drewsonne folinus b2jude karthiknew widdiix jonathan-fugue sandeeprajanp jinty naoyes binaryzer0 amitchmca ssala7 airmonitor pigmanbear gannino nijogeorgep cloudtech-arya arpanbee kuacuia askulkarni2 nicmaze jahnaviyerrapragada vladibb correasebastian tanduong elpasta aidinkhosrowshahi phpdude msato0731 jonahjon juwonom ifyoumakeit prashanth-m karthik199105 premgurung rodluiz cjpcloud j1anm1nxu karthickcse05 mk77-aws snegas kfatkin l1ahim ab-group markatwood johnguzmare einyx rafty rajtupakula dampsvamp123 smuniz-bairesdev michaelm-m vjr1990 lucasary anoopupadhyay francotel truerocha ppshein julian-chu jagannathsarma benluteijn anirudh-deshpande andreluiz365 acgoogle umaidashraf evrobert mohammedmv dannyfriz minasys hakuna-matatah ajhool kevinhuangnz agrawald alladarajendrakumar davidpanofsky tgosnell grichmond11 jpruizlu nkarwa

codepipeline-nested-cfn's Issues

CodePieline didn't allowing to update the stack

Hi Team,
I have created a lambda with the help of cloud formation [template.yml] file, Created one stack also. The lambda is in holding this stack itself.
Then after creating the new codePipeline also. The source is in GitHub, Build provider is AWS code build, Deploying is AWS cloud formation. What I created the previous, the stack only using it. But while running the pipeline stack is getting an error. already exist the lambda in this stack, like that.

I need exact requirements I a created some lambda with help of AWS cloud formation deploy commends,Now I need that lambda in code pipeline itself. From new pipeline I 'll create and update the lambda, what is the solution to without deleting the already created lambda?

Where is TEMPLATE_BUCKET set?

Hi, thank you for this work.

I was just wondering where TEMPLATE_BUCKET was set in the build file in the nested templates folder?

thanks

AMI does not exist

Is it possible for you to update the repo with a working AMI?

Sub-stack failure propagation

If a sub-stack fails to create, is there a way to have the error bubble upwards for capture, or sent somewhere? Currently, if a stack fails to create, the error reason is deleted with the stack during rollback. Kind of hard to troubleshoot unless you are staring at your pipeline.

Issues I ran into while running these stacks in a new account

Hi,

Great project! Here to report that I ran into several problems while running this stack on a new AWS account:

I had to create service-linked roles for autoscaling.amazonaws.com, elasticloadbalancing.amazonaws.com and rds.amazonaws.com services by running these commands in the CLI:

aws iam create-service-linked-role --aws-service-name autoscaling.amazonaws.com
aws iam create-service-linked-role --aws-service-name elasticloadbalancing.amazonaws.com
aws iam create-service-linked-role --aws-service-name rds.amazonaws.com

I'm aware that Amazon now creates these automatically when you provision a resource, but it's a bit faster this way.

I had to modify the AMI ID to the newest version of Amazon Linux 2 (ami-0e01ce4ee18447327 at the time of writing this post) in these files: config-test.json, config-uat.json, config-prod.json and env-details.py
I had to update the mysql version from 5.7.11 to 5.7.22 in databasestack.yml

Once these changes were made and the stack ran through the Test stage successfully, the UAT and Prod stages ran beautifully. Thank you for creating this, it was a great challenge for me to troubleshoot and I had a lot of fun doing it!

Geoff Moldenhauer

Is there no way to have stacks created by the pipeline deleted when the pipeline stack is deleted?

This is a question, not an issue per se.

i note this statement:

Note: While removing the resources, delete the Prod & UAT stacks created by pipeline before deleting the pipeline since those CloudFormations stacks uses the role created by pipeline stack.

I've experienced the pain of having to remove stacks in the right order.

When developing a pipeline you need to repeatedly create and delete the outlying codepipeline stack, and are left to delete the sub-stacks. It would be nice if there was a way to have stacks created by the pipeline be deleted when the pipeline gets deleted (in reverse dependency order).

Is there a way to do this, or does the user have to remember to delete the stacks created by the pipeline manually everytime, in the right order? Any advice is welcome.