Giter Club home page Giter Club logo

stemcell's Introduction

Stemcell Build Status

Stemcell launches instances in EC2. These instances are created to your specification, with knobs like AMI, instance type, and region exposed. The instances are bootstrapped with chef-solo, using a specified git repo and branch as the source of roles and recipes.

Installation

Add this line to your application's Gemfile:

gem 'stemcell'

And then execute:

$ bundle

Or install it yourself as:

$ gem install stemcell

Or build the gem locally and install it:

$ gem build stemcell.gemspec
$ gem install ./stemcell-0.4.3.gem

Configuration

If you're using the command line tool, you'll need to add a stemcell.json to the root of your chef repo. It contains default attributes for launching instances as well as the mapping from backing stores to images. For an example, see examples/stemcell.json.

You should create an RC file for stemcell with your standard options (and place it in the root dir as .stemcellrc?). You can see an example in examples/stemcellrc. As documented in that file, you will need:

  • a mono-repo for chef (like the kind described here)
  • a local checkout of the repo
  • an ssh key that's allowed to read that repo
  • AWS credentials with permission to create instances
  • an AWS ssh key pair
  • an encrypted data bag secret (optional)

Usage

Include your base config:

$ source ~/.stemcellrc

Simple launch:

$ stemcell $your_chef_role --git-branch $your_chef_branch

This will cause instance(s) to be launched and their ip's and instance id to be printed to the screen.

More options:

$ stemcell --help

Watching install:

$ stemcell $your_chef_role --tail

Terminating:

To terminate, use the AWS CLI and pass a space separated list of instance ids:

$ aws ec2 terminate-instances --instance-ids i-12345678 i-12345679 i-12345670

Automation

This README presents stemcell as a tool for administrators to use to create instances. However, we designed stemcell to be easily useful for automated systems which control server infrastructure. These automated systems can call out to stemcell on the command-line or use the ruby classes directly.

Similar Tools

There are a few additional tools which bootstrap EC2 instances with chef-solo. If you're using chef-server, obvious answer is knife-ec2. Unless you're working on a big team where lots of people edit cookbooks simultaneously, we strongly recommend this approach! It's especially excellent when paired with hosted chef, which makes getting off the ground with configuration management fast and easy.

If you want to use knife-ec2 with chef-solo, you could use knife solo. Another approach which is great for interactive usage involves using fabric to bootstrap chef(with gist).

Finally, we couldn't resist doing a bit of code archeology. People have been using chef with EC2 for a long time! One early article is this one, which isn't even on the web anymore. However, it's spawned some recently-active tools like this and this. Similar approaches are mentioned here, with code here or here (with accompanying blog post) This article, also mentions many worthwhile predecessors.

stemcell's People

Contributors

alexism avatar brndnmtthws avatar christofdamian avatar clizzin avatar darnaut avatar igor47 avatar jacknagz avatar jacobwgillespie avatar jimmyngo avatar josephsofaer avatar jtai avatar kunickiaj avatar liukai avatar martinrhoads avatar melaniecebula avatar nelgau avatar noggi avatar patrickviet avatar ramyak avatar ryxias avatar sandstrom avatar vvassiliouk avatar wmoss avatar ziliangpeng avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

stemcell's Issues

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

  spec.license = 'MIT'
  # or
  spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

Replacement Discussion

It seems like AirBnB doesn't use stemcell internally anymore.

It would be interesting to learn what other solution you've switched to (internal or some other open-source mechanism?).

Since this repo is somewhat stalled now, perhaps you could also add a note about this in the readme, explaining in which cases stemcell is still useful, and when people may want to look elsewhere (and at what).

What do you think @darnaut?

Cloud-init support โ˜

Has there been any discussions on supporting cloud-init data, along with the chef setup script?

It is possible to supply both cloud init data (YAML) and a script via a mime multi-part archive.

Cloud init configs are useful to e.g. set the hostname, adding CA certs, post setup phone home and many other things. Although some of this is configurable via chef too, some of them are individual to the machine (e.g. setting unique hostnames), and hence are trickier to handle via chef recipes.

Fix running under Chef 14.x

I haven't been able to get Chef running under 14.x, due to the following error:

Errno::ENOENT: No such file or directory @ rb_sysopen - /.chef/chef_guid
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/client.rb:1012:in `initialize'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/client.rb:1012:in `open'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/client.rb:1012:in `generate_guid'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/client.rb:262:in `run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/application.rb:303:in `run_with_graceful_exit_option'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/application.rb:279:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/application.rb:261:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/application/solo.rb:304:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/lib/chef/application/solo.rb:227:in `run'
/opt/chef/embedded/lib/ruby/gems/2.5.0/gems/chef-14.1.1/bin/chef-solo:25:in `<top (required)>'
/usr/bin/chef-solo:71:in `load'
/usr/bin/chef-solo:71:in `<main>'

Everything runs just fine under 13.x. Reading through the 14.x breaking changes there isn't anything that immediately pops out.

There were issues with 13.x too, but I managed to solve (#94) that.

Helping to do additional Chef runs

The cloud-init bootstrap does a fine job with the first Chef run, but I'm thinking how to do sub-sequential Chef runs. So far I've done the following manually:

  1. Create /tmp/chef.json file with the bootstrap data: {"role": "xxx", "env": "production", "branch": "production"}
  2. eval ssh-agent
  3. ssh-add git_key
  4. "git pull origin production" to update the Chef repository
  5. chef-solo -j /tmp/chef.json -o role[xxx]

This can easily be compiled into a script. As the bootstrap.sh doesn't save the chef json file into a persistent file (it saves it into a tempfile) I would need to modify bootstrap.sh to save it for example into /etc/chef/chef.json, so that I can use the same attributes as what was used on the initial chef run.

If I do that I can just as easily modify bootstrap.sh to create this kind of file under (let say) /usr/local/bin which would do this.

What do you think?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.