Giter Club home page Giter Club logo

simple_deploy's Introduction

Build Status

Simple Deploy is in maintenance mode!

We will continue to provide support for bug fix requests, however new features will not be added.

It has been a good run, but it has now come to an end. When Simple Deploy was started, the AWS CLI had minimal Cloud Formation support; that is no longer the case. The AWS CLI is mature, well-documented and can be used to perform the majority of the Cloud Formation actions provided by Simple Deploy and we believe new customers will be better served leveraging it for their Cloud Formation stack management.

For the features which Simple Deploy provides that are not available via AWS CLI, we have created the following utilities to provide near-like services, to what is provided by Simple Deploy, which can be integrated with the AWS CLI.

  • cfn-clone allows for cloning Cloud Formation stacks. It will leverage the AWS CLI to read the parameters and template from an existing stack. It allows you to override either the template or inputs and create a new stack with the updated attirbutes and template.

  • heirloom-url generates URLs that point to resources which have been uploaded to Heirloom. This can be coupled with the AWS CLI to update the app or chef_repo.

For example, to update the app and chef_repo of a stack, and then kick off a command, similiar to the Simple Deploy deploy subcommand, you could use the following bash script.

#!/bin/bash

app_id=$1
chef_id=$2
 
app_url=`heirloom-url -bucket-prefix=bp -domain=my-app -encrypted=true -id=$app_id -region=us-west-2`
chef_url=`heirloom-url -bucket-prefix=bp -domain=my-chef -encrypted=true -id=$chef_id -region=us-west-2`

aws cloudformation update-stack --stack-name my-app-stack \
                                --parameters AppArtifactURL=$app_url,ChefRepoURL=$chef_url
 
ips=`aws ec2 describe-instances --filter Name=Name,Values=my-app-stack \
| jq --raw-output '.Reservations[].Instances[].PublicIpAddress'`
 
for ip in $ips; do
  ssh ip 'chef-solo -c /var/chef/config/solo.rb -o role[app]โ€™
done

If you are interested in more efficient SSH parallelization, I would look into one of the below tools which could be integrated into the above script.

Overview

Simple Deploy is an opinionated gem that helps manage and perform directed deployments to AWS Cloud Formation Stacks.

Prerequisites

  • Ruby version 1.9.2 or higher installed.
  • AWS account access key and secret key.

Installation

Install the gem

gem install simple_deploy --no-ri --no-rdoc

Create a file ~/.simple_deploy.yml and include within it:

environments:
  preprod:
    access_key: XXX
    secret_key: yyy
    region: us-west-1

Documentation

For more information, please view the Simple Deploy Wiki.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

simple_deploy's People

Contributors

bradly avatar bw-intuit avatar capen avatar ccloes-intuit avatar dylanvaughn avatar grantleehoffman avatar jonchay avatar keviny22 avatar philiplrb avatar shadnygren avatar thbishop avatar weavenet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

simple_deploy's Issues

Add a clone stack command

  • It should allow users to pass the old stack name and a new stack name and clone the core stack attributes
  • Extra attributes are added at later times via deploys and updates and those should not be cloned

Fix simple deploy to exit gracful when a stack has no running instances.

I, [2012-07-19T15:42:02.658937 #35493] INFO -- : Starting deployment.
/Users/bweaver/.rvm/gems/ruby-1.9.3-p125@simple_deploy/gems/capistrano-2.12.0/lib/capistrano/configuration/connections.rb:163:in execute_on_servers':simpledeploy' is only run for servers matching {}, but no servers matched (Capistrano::NoMatchingServersError)

Display a helpful message if an argument is missing

For example when performing a create without a template:

#simple_deploy create -e preprod_shared_us_west_1 -n lc-pod-1-dev-1-adminpublicip-1
/Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_creater.rb:28:in `initialize': can't convert nil into String (TypeError)
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_creater.rb:28:in `open'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_creater.rb:28:in `read_template_from_file'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_creater.rb:11:in `initialize'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack.rb:87:in `new'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack.rb:87:in `stack_creater'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack.rb:24:in `create'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/lib/simple_deploy/stack.rb:18:in `create'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/lib/simple_deploy/cli.rb:123:in `start'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/bin/simple_deploy:6:in `<top (required)>'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `load'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `<main>'

Incorrect PRIMARY_HOST when deploying without an ssh host

This injects the public ip. Shouldn't it be the private one?

2012-07-26 16:42:22 -0700 INFO : Deploy command: 'env CHEF_REPO_URL=s3://intu-lc-us-west-1/live_community_chef_repo/e5c95bd4eb745da7529ab2affe0924de00d20933.tar.gz \
COOKBOOKS_URL=s3://intu-artifacts-us-west-1/cookbooks/643296e8e98dfd3529daf0b539ff9dbc063ec5b2.tar.gz \
APP_URL=s3://intu-lc-us-west-1/live_community/71c24735844c97be12b04bdf8a12d9e73f24f116.tar.gz \
PRIMARY_HOST=204.236.179.235 /opt/intu/admin/bin/configure.sh'.

Status is blowing up

On 4.1

tbishop in ~ (12:15:56 || 1.9.3-p125 || master*)
#simple_deploy status -e preprod_shared_us_west_1 -n lc-pod-1-dev-1-admininstance-1
/Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.4.1/lib/simple_deploy/stack.rb:99:in `status': private method `status' called for #<Stackster::Stack:0x007fe2c0988c30> (NoMethodError)
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.4.1/lib/simple_deploy/cli/status.rb:33:in `show'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.4.1/lib/simple_deploy/cli.rb:49:in `start'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.4.1/bin/simple_deploy:6:in `<top (required)>'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `load'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `<main>'

Stackster should return a valid error when no 'stacks' domain exists

The below should only show stacks for test

simple_deploy list -e test

lc-pod-1-e2e-1-db-traffic-container
lc-pod-1-e2e-1-elb-traffic-container
lc-pod-1-e2e-1-admin-traffic-container
lc-pod-1-e2e-1-nat-traffic-container
lc-pod-1-e2e-1-admin-1
lc-pod-1-e2e-1-db-master-1
lc-pod-1-e2e-1-elb-2
lc-pod-1-e2e-1-nat-1
lc-pod-1-e2e-1-elb-1
lc-pod-1-dev-1-db-traffic-container
lc-pod-1-dev-1-elb-traffic-container
lc-pod-1-dev-1-admin-traffic-container
lc-pod-1-dev-1-nat-traffic-container
lc-pod-1-dev-1-db-master-1
lc-pod-1-dev-1-nat-1
lc-pod-1-dev-1-elb-1
lc-pod-1-dev-1-elb-2
lc-pod-1-dev-1-admin-1
lc-pod-1-dev-1-cache-traffic-container
lc-pod-1-dev-1-cache-elb-traffic-container
lc-pod-1-e2e-1-cache-traffic-container
lc-pod-1-e2e-1-cache-1
lc-pod-1-e2e-1-cache-elb-traffic-container
lc-pod-1-dev-1-app-2
lc-pod-1-dev-1-cache-1
lc-pod-1-e2e-1-app-1
lc-pod-1-dev-1-worker-1
lc-pod-1-e2e-1-worker-1

Handle case when attributes are already up to date

Specifying an update to attributes which are already set currently throws an exception. The relevant portion:

  response => #<Excon::Response:0x007fe962ac9c90 @body="<ErrorResponse xmlns=\"http://cloudformation.amazonaws.com/doc/2010-05-15/\">\n  <Error>\n    <Type>Sender</Type>\n    <Code>ValidationError</Code>\n    <Message>No updates are to be performed.</Message>\n  </Error>\n  <RequestId>77da56ae-c1b3-11e1-a106-23d01f7657e2</RequestId>\n</ErrorResponse>\n", @headers={"x-amzn-RequestId"=>"77da56ae-c1b3-11e1-a106-23d01f7657e2", "Content-Type"=>"text/xml", "Content-Length"=>"288", "Date"=>"Fri, 29 Jun 2012 06:27:15 GMT"}, @status=400>

Add a check for deploy attributes

It would be nice if simple_deploy reported missing attribute dependencies rather than throwing a stack trace. This could be done around the same time it checks the lock. The attributes to check are the bucket_prefix and domain attributes.

Post message to Campfire on Deployments

I'm thinking something more flexible, maybe a plugins or handlers style directory where people could put arbitrary ruby code to perform different actions...

Could simple_deploy post to Campfire after deploying? Maybe the message could include the deployer, environment, and SHA. And for bonus points, the SHA could be a link to the commit in GitHub. :)

Update subcommand should accept a force option

Now that we are configuring deploys through attributes, it is easy to run into the situation that locks you out of deploys.

  • Run deploy and it fails because the attributes aren't set
  • Run update and it fails because the stack is locked for deployment
  • User needs a force option so they can get out of this situation themselves

Add ability to tag instances for easier access.

To keep engineers from need to remember long stack names (lc-pod-2-prod-1-app-1,etc), we would like to tag instances with friendly tags.

"lc-pod-2-prod-1-app-1" could be given a friendly tag "prod-app" which engineer could use to deploy against.

Version isn't working

On 4.1

tbishop in ~ (12:16:26 || 1.9.3-p125 || master*)
#simple_deploy -v
Unknown command: '-v'.
simple_deploy [attributes|create|destroy|environments|events|instances|list|template|outputs|parameters|resources|ssh|status|update] [options]
Append -h for help on specific subcommand.

tbishop in ~ (12:16:59 || 1.9.3-p125 || master*)
#simple_deploy --version
Unknown command: '--version'.
simple_deploy [attributes|create|destroy|environments|events|instances|list|template|outputs|parameters|resources|ssh|status|update] [options]
Append -h for help on specific subcommand.

tbishop in ~ (12:17:06 || 1.9.3-p125 || master*)
#simple_deploy -V
Unknown command: '-V'.
simple_deploy [attributes|create|destroy|environments|events|instances|list|template|outputs|parameters|resources|ssh|status|update] [options]
Append -h for help on specific subcommand.

Unable to accept attributes with csv values

There are items in cloud formation templates which require csv parameters. It looks like the attribute parsing can't handle this situation.

If you provide:

simple deploy .... -a 'MaximumAppInstances=1,MinimumAppInstances=1,LoadBalancerIds="ELB-W5T24H1HECA0,ELB-VQX6F1KUPMR2"'

It reads it as:

MaximumAppInstances : 1
MinimumAppInstances : 1
LoadBalancerIds : ELB-W5T24H1HECA0
VQX6F1KUPMR2 : VQX6F1KUPMR2

I also tried a couple of different quoting methods without success

Add en explicit lock environment from deployment flag

This can not be over riden by force, but will need to be unset before some can deploy. For example

simple_deploy lock -e xxx -n yyyy

....
simple_deploy deploy -e xxx -n yyyy
error: Environment locked by bweaver
....

simple_deploy unlock -e xxx -n yyyy

Deployment throws exception when sending a message if .simple_deploy not up to date.

2012-07-24 10:50:03 -0700 INFO : Deploy completed succesfully for lc-pod-1-prod-1-app-2.
/Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/notifier.rb:29:in send': undefined methodkeys' for nil:NilClass (NoMethodError)
from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/notifier.rb:25:in send_deployment_complete_message' from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/cli/deploy.rb:47:inblock in deploy'
from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/cli/deploy.rb:35:in each' from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/cli/deploy.rb:35:indeploy'
from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/lib/simple_deploy/cli.rb:33:in start' from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/gems/simple_deploy-0.4.2/bin/simple_deploy:6:in<top (required)>'
from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/bin/simple_deploy:19:in load' from /Users/jarmitage/.rvm/gems/ruby-1.9.3-p125@mvp/bin/simple_deploy:19:in

'

Simple Deploy destroy command outputs deployment logs

Is it running it?

[root@ip-10-170-250-9 ~]# simple_deploy destroy -n image02 -e infrastructure_us-west-1
2012-08-07 14:27:14 -0400 INFO : Creating deployment to image02.
2012-08-07 14:27:14 -0400 INFO : Not using an ssh gateway.
2012-08-07 14:27:14 -0400 INFO : Deploy command: 'env CHEF_REPO_URL=s3://intu-lc-us-west-1/live_community_chef_repo/.tar.gz COOKBOOKS_URL=s3://intu-artifacts-us-west-1/cookbooks/.tar.gz APP_URL=s3://intu-lc-us-west-1/live_community/.tar.gz PRIMARY_HOST= /opt/intu/admin/bin/configure.sh'.
2012-08-07 14:27:15 -0400 INFO : Cloud Formation stack destroy completed.
2012-08-07 14:27:15 -0400 INFO : Delete from SimpleDB successful.
2012-08-07 14:27:15 -0400 INFO : image02 destroyed.

environments command blows up

tbishop in ~ (10:35:59 || 1.9.3-p125 || master*)
#simple_deploy environments
/Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/config.rb:31:in `initialize': No such file or directory - /Users/tbishop/.stackster.yml (Errno::ENOENT)
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/config.rb:31:in `open'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/config.rb:31:in `load_config_file'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/config.rb:7:in `initialize'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_lister.rb:5:in `new'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/stackster-0.2.4/lib/stackster/stack/stack_lister.rb:5:in `initialize'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/lib/simple_deploy/cli.rb:104:in `new'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/lib/simple_deploy/cli.rb:104:in `start'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/gems/simple_deploy-0.3.5/bin/simple_deploy:6:in `<top (required)>'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `load'
    from /Users/tbishop/.rvm/gems/ruby-1.9.3-p125/bin/simple_deploy:23:in `<main>'

tbishop in ~ (10:36:08 || 1.9.3-p125 || master*)
#simple_deploy -v
0.3.5

tbishop in ~ (10:36:22 || 1.9.3-p125 || master*)
#

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.