Giter Club home page Giter Club logo

agent's Introduction

Buildbox Agent Build status

There are three commands included in the buildbox-agent package:

  • buildbox-agent - the main job runner, which polls Buildbox for build steps to execute
  • buildbox-data - reads and writes to your build-wide key/value store
  • buildbox-artifact - uploads and downloads files to your build-wide file store

Installing

Simply run the following command (see the source), which will automatically download the correct binaries for your platform (or if you'd prefer not to run this install script see the manual installation guide):

bash -c "`curl -sL https://raw.githubusercontent.com/buildbox/agent/master/install.sh`"

Copy the access token from the agent settings page on Buildbox:

image

And then start the agent with the token:

~/.buildbox/buildbox-agent start --access-token b9c784528b92d7e904cfa238e68701f1

Now you're all set to run your first build!

If you're using Windows, you'll want to use the Ruby agent.

Launching on system startup

We've some templates for the default process manageers for various platforms. Using a process manager will allow to you ensure buildbox-agent is running on system boot, and will allow for easy upgrades and restarts because you can simply kill the process and the process manager to start it up again.

If you have another to contribute, or need a hand, let us know! (pull requests also welcome)

Upgrading

Upgrading the agent is simply a matter of re-running the install script and then restarting the agent (with a USR2).

  1. Run the install script again. This will download new copies of the binaries. Dont' worry, it won't override your bootstrap.sh file.

    bash -c "`curl -sL https://raw.githubusercontent.com/buildbox/agent/master/install.sh`"
  2. Tell the buildbox-agent process to restart by sending it an USR2 signal

    killall -USR2 buildbox-agent

    This will tell it to finish off any current job, and then shut itself down.

  3. If you use a process monitor such as upstart or launchd it will startup again automatically, with no more work required. If you don't, just start the buildbox-agent like you did initially.

  4. Check your buildbox-agent logs to make sure it has started up the agent again.

If you're running a really long job, and just want to kill it, send the USR2 signal twice. That'll cause the buildbox-agent process to cancel the current job, and then shutdown.

How it works

After starting, buildbox-agent polls Buildbox over HTTPS looking for work.

When a new job is found it executes bootstrap.sh with the standard Buildbox environment variables and any extra environment variables configured in your build pipeline's steps.

As the build is running the output stream is continously sent to buildbox, and when the build finishes it reports the exit status and then returns to looking for new jobs to execute.

Using a bootsrap.sh ensures that buildbox web can't be configured to run arbitrary commands on your server, and it also allows you to configure bootsrap.sh to do anything you wish (although it works out-of-the-box with git-based projects).

Artifact uploads

Uploading artifacts is handled by a seperate tool buildbox-artifact which is bundled with the agent. You can see it's general usage in templates/bootstrap.sh.

Uploading Artifacts to your own S3

If you'd like to upload artifacts to your own Amazon S3 bucket, edit your bootstrap.sh file, and replace the buildbox-artifact call with something like this:

export AWS_SECRET_ACCESS_KEY=yyy
export AWS_ACCESS_KEY_ID=xxx
# export AWS_DEFAULT_REGION=ap-southeast-2 (Optionally set the region. Defaults to us-east-1)
buildbox-artifact upload "$BUILDBOX_ARTIFACT_PATHS" "s3://name-of-your-s3-bucket/$BUILDBOX_JOB_ID" --endpoint $BUILDBOX_AGENT_ENDPOINT

We'll recognise the s3:// prefix and upload the artifacts to S3 to the bucket name name-of-your-s3-bucket (obviously you'll want to change it to the name of your own S3 bucket)

If you upload artifacts to your own S3 Bucket, you can further secure your artifacts by Restricting Access to Specific IP Addresses

Customizing bootstrap.sh

You can think of the bootstrap.sh file as a pre-script for your build. The template that we provide will checkout a Git project, and run your build script, but that's just the default. You can make it do what ever you like. If you don't use Git, you can edit the script to use what ever version control software you like.

You can also use bootstrap.sh to control what repositores are checked out on your CI server. For example, you could add something like this to the top of your bootstrap.sh file:

if [ "$BUILDBOX_REPO" != *github.com/keithpitt/my-app* ]
then
  echo "Unrecognised repo: $BUILDBOX_REPO"
  exit 1
fi

The benefit of the bootstrap.sh is that it's written in bash. You can change it how ever you like and customize how builds get run on your servers.

Manual Installation

Here we'll show you how to manually install the buildbox agent.

  1. Create a folder at ~/.buildbox

    mkdir -p ~/.buildbox
  2. Download the correct binaries for your platform. See: https://github.com/buildbox/agent/releases/tag/v0.2 for a list for binaries.

    wget https://github.com/buildbox/agent/releases/download/v0.2/buildbox-agent-linux-amd64.tar.gz
  3. Extract the tar. This should extract buildbox-agent and buildbox-artifact to the ~/.buildbox folder.

    tar -C ~/.buildbox -zvxf buildbox-agent-linux-amd64.tar.gz
  4. Download our example bootstrap.sh and put it in ~/.buildbox

    wget -q https://raw.githubusercontent.com/buildbox/agent/master/templates/bootstrap.sh -O ~/.buildbox/bootstrap.sh
  5. (Optional) Add ~/.buildbox to your $PATH so you can access the binaries eaiser.

    echo 'export PATH="$HOME/.buildbox:$PATH"' >> ~/.bash_profile

Development

Some basic instructions on setting up your Go environment and the codebase for running.

# Make sure you have go installed.
brew install go --cross-compile-common
brew install mercurial

# Setup your GOPATH
export GOPATH="$HOME/Code/go"
export PATH="$HOME/Code/go/bin:$PATH"

# Install godep
go get github.com/kr/godep

# Checkout the code
mkdir -p $GOPATH/src/github.com/buildbox/agent
git clone [email protected]:buildbox/agent.git $GOPATH/src/github.com/buildbox/agent
cd $GOPATH/src/github.com/buildbox/agent
godep get
go run *.go start --token xxx --debug

To test the commands locally:

go run cmd/artifact/artifact.go upload "buildbox/*.go" --agent-access-token=[..] --job [...] --debug
go run cmd/artifact/artifact.go download "buildbox/*.go" . --agent-access-token=[..] --job [...] --debug

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Copyright

Copyright (c) 2014 Keith Pitt, Buildbox. See LICENSE for details.

agent's People

Contributors

donpinkster avatar keithpitt avatar kouky avatar pda avatar sj26 avatar theojulienne avatar tommeier avatar toolmantim avatar wolfeidau avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.