Giter Club home page Giter Club logo

ironfan-ci's Introduction

Ironfan-CI: Continuous Testing for your Architecture

Ironfan, the foundation of the Infochimps Platform, is an expressive toolset for constructing scalable, resilient architectures. It works in the cloud, in the data center, and on your laptop, and it makes your system diagram visible and inevitable. Inevitable systems coordinate automatically to interconnect, removing the hassle of manual configuration of connection points (and the associated danger of human error). For more information about Ironfan and the Infochimps Platform, visit infochimps.com.

This repo implements continuous testing for your architecture:

  • cookbooks to set up a local chef server and jenkins master
  • vagrant definitions for the operating systems ironfan supports
  • test framework that combines cucumber to verify the silverware announcement contract
  • (soon) hooks to tie these CI tests to your monitoring system

Getting Started

To jump right into using Ironfan, follow our Installation Instructions. For an explanatory tour, check out our Hadoop Walkthrough. Please file all issues on Ironfan Issues.

Index

Ironfan-CI works together with the full ironfan toolset:

Core Tools:

  • ironfan-homebase: Centralizes the cookbooks, roles and clusters. A solid foundation for any Chef user.
  • ironfan gem: The core Ironfan models, and Knife plugins to orchestrate machines and coordinate truth among your homebase, cloud and chef server. It comes with ironfan-homebase.
  • ironfan-pantry: Our collection of industrial-strength, cloud-ready recipes for Hadoop, HBase, Cassandra, Elasticsearch, Zabbix and more.
  • silverware cookbook: Helps you coordinate discovery of services ("list all the machines for awesome_webapp, that I might load balance them") and aspects ("list all components that write logs, that I might logrotate them, or that I might monitor the free space on their volumes"). Found within the ironfan-pantry.

Core Documentation:

Note: Ironfan is not compatible with Ruby 1.8. All versions later than 1.9.2-p136 should work fine.


in progress: some of the below is speculative (@sya)


You can use VeeWee, Vagrant and Virtualbox to make chef development ridiculously funner.

Testing homebase

Just need the vagrants, no knife or anything --

  • chef server username is runner username

  • looks in ~/.chef/{username}.pem for user key

  • looks in ~/.chef/{organization}-testing-credentials/client_keys/client-{node_name}.pem for chef client key

  • To populate chef node,

    knife cluster sync sandbox --no-cloud --sync-all
    

Building boxes

  • follow instructions in ironfan_ci/README.md

Building vagrants with knife

Script Integration:

require 'vagrant'

# Example of emulating vagrant up with some code around it
task :up do
  puts "About to run vagrant-up..."
  env = Vagrant::Environment.new(:cwd => "/path/to/my/project")
  env.cli("up")
  puts "Finished running vagrant-up"
end

task :graceful_down do
  env = Vagrant::Environment.new(:cwd => "/path/to/my/project")
  env.vms.each do |vm|
    vm.channel.sudo("halt")
  end
end

These are the parts of the vagrant automation:

  • convert cluster definition to Vagrantfile
  • knife provisions vagrant servers
  • a set of conventions for using Vagrant with chef in a local debugging or testing environment.

on desktop:

  • ssh key for machine

on desktop and chef server:

  • server.rb
  • validation.pem
  • validator.pem
  • cookbooks
  • roles

on desktop and chef client:

  • chef-dna.json
  • client.pem
  • validator.pem
  • client.rb
  • data bag keys

Here's what you need:

  • jenkins server, properly installed and configured

  • worker machines:

    • jenkins worker
    • discoverable by jenkins
    • virtualbox, vagrant etc
  • a testing chef server

  • vagrants that are clients of the testing chef server

Decisions

you can either go

A: [jenkins] -> on worker -> vagrant up, vagrant ssh [chef-client runner], vagrant rollback

B: [jenkins] -> vagrant up (becomes a jenkins ssh worker) -> jenkins ssh [chef-client runner]

I went with A, because it lends itself to a distributed setup; doesn't have the potential confounding effects of the jenkins ssh-ness; and is easy for me to simulate and debug

FIXME

  • there are OSX things in odd places within jenkins
  • there are lots of roles running about inside chimpstation

Troubleshooting

  • make sure you have remote login on for all users -- visit the 'Sharing' System Prefences to correct this:

    sshd: fatal: Access denied for user _jenkins by PAM account configuration
    

ironfan-ci's People

Contributors

sya avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.