Giter Club home page Giter Club logo

ecs-blue-green-deployment's Introduction

NOTE: Same reference architecture, but using AWS Fargate is available in fargate branch

Blue/Green deployments on ECS

This reference architecture is in reference to blog post on blue green deployments on ECS. It creates a continuous delivery by leveraging AWS CloudFormation templates. The templates creates resources using Amazon's Code* services to build and deploy containers onto an ECS cluster as long running services. It also includes a manual approval step facilitated by lambda function that discovers and swaps target group rules between 2 target groups, promoting the green version to production and demoting the blue version to staging.

Pre-Requisites

This example uses AWS Command Line Interface to run Step-3 below.

Please follow instructions if you haven't installed AWS CLI. Your CLI configuration need PowerUserAccess and IAMFullAccess IAM policies associated with your credentials

aws --version

Output from above must yield AWS CLI version >= 1.11.37

Quick setup in three steps

1. Fork ECS Sample app

Fork the Amazon ECS sample app GitHub repository into your GitHub account.

Clone the ECS Sample app repo

git clone https://github.com/<your_github_username>/ecs-demo-php-simple-app

2. Clone ECS blue green repo

git clone https://github.com/awslabs/ecs-blue-green-deployment

2a (Optional) . Switch to fargate branch , if you want to use AWS Fargate

git checkout fargate

3. Run bin/deploy

bin/deploy

Here are the inputs required to launch CloudFormation templates:

  • S3 Bucket: Enter S3 Bucket for storing your CloudFormation templates and scripts. This bucket must be in the same region where you wish to launch all the AWS resources created by this example.
  • CloudFormation Stack Name: Enter CloudFormation Stack Name to create stacks
  • GitHubUser: Enter your GitHub Username
  • GitHubToken: Enter your GitHub Token for authentication (https://github.com/settings/tokens)

Sit back and relax until all the resources are created for you. After the templates are created, you can open ELB DNS URL to see the ECS Sample App

For testing Blue Green deployment, Go ahead and make a change in ECS Sample App. For ex, edit src/index.php and update the background-color to #20E941 to change to Green background color. After commiting to your repo, Code Pipeline will pick the change automatically and go through the process of updating your application.

Click on "Review" button in Code pipeline management console and Approve the change. Now you should see the new version of the application with Green background.

Resources created in this exercise

Count AWS resources
7 AWS CloudFormation templates
1 Amazon VPC (10.215.0.0/16)
1 AWS CodePipeline
2 AWS CodeBuild projects
1 Amazon S3 Bucket
1 AWS Lambda
1 Amazon ECS Cluster
2 Amazon ECS Service
1 Application Load Balancer
2 Application Load Balancer Target Groups

Implementation details

During first phase, the parent template (ecs-blue-green-deployment.yaml) kicks off creating VPC and the resources in deployment-pipeline template. This creates CodePipeline, CodeBuild and Lambda resources. Once this is complete, second phase creates the rest of resources such as ALB, Target Groups and ECS resources. Below is a screenshot of CodePipeline once all CloudFormation templates are completed

codepipeline

The templates create two services on ECS cluster and associates a Target Group to each service as depicted in the diagram. Blue Target Group is associated with Port 80 that represents Live/Production traffic and Green Target Group is associated with Port 8080 and is available for new version of the Application. The new version of the application can be tested by accessing the load balancer at port 8080, example http://LOAD_BALANCER_URL:8080 .If you want to restrict the traffic ranges accessing beta version of the code, you may modify the Ingress rules here. During initial rollout, both Blue and Green service serve same application versions.As you introduce new release, CodePipeline picks those changes and are pushed down the pipeline using CodeBuild and deployed to the Green service. In order to switch from Green to Blue service (or from beta to Prod environment), you have to Approve** the release by going to CodePipeline management console and clicking Review** button. Approving the change will trigger Lambda function (blue_green_flip.py) which does the swap of ALB Target Groups. If you discover bugs while in Production, you can revert to previous application version by clicking and approving the change again. This in turn will put Blue service back into Production. To simplify identifying which Target Groups are serving Live traffic, we have added Tags on ALB Target Groups. Target Group IsProduction Tag will say true for Production application.

bluegreen

Here is further explaination for each stages of Code Pipeline.

During Build stage

  • During first phase, CodeBuild builds the docker container image and pushes to Amazon ECR.

  • During second phase, Codebuild executes scripts/deployer.py which executes the following scripted logic

    1. Retrieve artifact (build.json) from the previous phase (CodeBuild phase, which builds application container images)
    2. Check if the load balancer exists. Name of the ELB is fed through environment variable by the pipeline.
    3. Get tag key value of the target group, running on port 8080 and 80 with KeyName as "Identifier". It will be either "Code1" or "Code2"
    4. Get Sha of the image id running on target group at port 8080 and 80
    5. Edit the build.json retrieved from step-1 and append the values retrieved in step3 and step4
    6. Save the modified build.json. This file is the output from codebuild project and fed as an input to the CloudFormation execution stage.This json file has the following schema { "Code1" : "CONTAINER_TAG1", "Code2" : "CONTAINER_TAG2" } If the load balancer does not exists (as found in step-2), this would imply that the stack is executed for the first time, and the values of "CONTAINER_TAG1" and CONTAINER_TAG2" will be the same and default to the value retrieved from build.json in step-1

During Deploy stage CodePipeline executes templates/ecs-cluster.yaml. The CloudFormation input parameters with KeyName as "Code1" and "Code2" are overwritten with the values as written in the build.json, retrieved from the second phase of Build Stage.

During Review stage The pipeline offers manual "Review" button so that the approver can review code and Approve new release. Providing approvals at this stage will trigger the Lambda function (blue_green_flip.py) which swaps the Green Target Group to Live traffic. You can checkout sample app to see new release change. blue_green_flip.py has the following logic scripted

  1. Read Job Data from input json
  2. Read Job ID from input json
  3. Get parameters from input json
  4. Get Load balancer name from parameters
  5. Identify the TargetGroup running on this Load Balancer at port 80 and port 8080. Perform the TargetGroup Swap. Also swap the values of "IsProduction" tags.
  6. Send success or failure to CodePipeline

Cleanup

First delete ecs-cluster CloudFormation stack, this will delete both ECS services (BlueService and GreenService) and LoadBalancer stacks. Next delete the parent stack. This should delete all the resources that were created for this exercise

ecs-blue-green-deployment's People

Contributors

anshrma avatar dalbhanj avatar jicowan avatar ysosiak avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ecs-blue-green-deployment's Issues

Use AWS ECR as source

First of all, thank you for creating this project, which gave me a great starting point!

Instead of using Github as the source of my pipeline, we build the container images in another system and push the images to ECR. Therefore, we need CodePipeline to pick up changes in ECR and run deploy the latest image.

Bug: Python runtime in deployment-pipeline.yaml - BlueGreenFlipLambda Resource outdated

Hi there,

cloning this repo and following the instructions on your workshop page (https://cicd-pipeline-cfn-ecs-fargate-bluegreen.workshop.aws/en/cicd/script.html) results in a stack creation error of the DeploymentPipeline stack with the following error:

"Resource handler returned message: "The runtime parameter of python2.7 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (python3.9) while creating or updating functions. (Service: Lambda, Status Code: 400, Request ID: 96df5a45-faa2-4641-9b6f-31b401622c1d, Extended Request ID: null)" (RequestToken: 1ae75b77-6c07-1dc6-0fc9-1483a5033e45, HandlerErrorCode: InvalidRequest)"

It seems that the used runtime for the lambda (python 2.7) is not supported by cloudformation anymore

No revert button

The documentation indicates

If you discover bugs while in Production, you can revert to previous application version by clicking and approving the change again.

Does this imply you need to revert code and re-commit to have the pipeline build again, or is there a way to simply flip the blue/green deployments back to the previous version. The later would be preferable. In this case would a seperate pipeline which skips the code build step and only performs the flip operation be required?

What command

What is the command in step 3 to run bin?

  1. Run bin/deploy

Blue/green strategy for applications with port on 443

Thanks for the reference example. The post provides a Blue/green strategy of using different ports and listeners to control the traffic between blue and green. In enterprises like ours, we use products for authentication and they can work only in certain ports like 443. What could be the blue/green strategy in such cases?
We have an apache container running on 443 linked to a springboot container that is running on 8080. Only port 443 can be used since the tool (Cleartrust) used for authentication require that port.

S3 permissions error

Hi,

I realise this is a fairly old demo now but I was surprised when it didn't work out of the box.

I tried to create the stack on the fargate branch in eu-west-2 so had to update the TemplateURL fields to:

https://${TemplateBucket}.s3.eu-west-2.amazonaws.com/templates/vpc.yaml

This meant the stack creation could progress further, but still failed to create the DeploymentPipeline stack with the following error for the LoadBalancer resource:

S3 error: Access Denied For more information check http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html

I then added s3:* to the inline policy within CloudFormationExecutionRole, but I am still seeing the above error.

Can anyone verify that they can get this stack working?

Thank you

David

No Autoscaling

As described in my posts at https://forums.aws.amazon.com/thread.jspa?threadID=274249 and https://stackoverflow.com/questions/48917361/autoscaling-blue-green-ecs-clusters, we built a blue-green cluster on Terraform inspired by this sample, but we have not been able to get autoscaling working. We have been able to get EC2 instances added to the cluster, but the Blue and Green services do not spin up additional tasks to make use of these instances.

It would be great if this sample were expanded to provide working code for autoscaling.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.