Giter Club home page Giter Club logo

mytardis-chef's Introduction

Overview

Written by Steve Androulakis (github user steveandroulakis) on 9th August 2012, updated 30th April 2013

This Cookbook installs the current master of MyTardis - http://github.com/mytardis/mytardis

It's been tested and confirmed to work on Ubuntu 10 (Lucid), 12 (see notes below), and CentOS 6.

It's also been tested using Vagrant and the NeCTAR Research Cloud (see http://nectar.org.au/).

Download or git clone this cookbook and use it with Chef Solo (see https://github.com/mytardis/mytardis-chef/wiki/Chef-Solo-Guide) or upload it to Hosted Chef (look on Chef's site for guides).

This installation is very minimal (no accounts, no example data) but is enough to get you started.

Health Warning

If you use this recipe for building a "production" MyTardis instance (i.e. one where the data matters), then you need to be aware of a couple of things:

  1. This recipe does not set up backups of either the MyTardis database or the data store area weher data files are kept. You need to make your own arrangements.

  2. MyTardis uses South migration for managing database schema changes, and this recipe in its current form will apply any pending South migrations without any warning. This ''should'' work, but there is always a risk that the migration will go wrong, and that you will be left with a corrupted database. It is prudent to ''back up your database and data'' before you attempt to deploy a new version.

  3. This recipe works by checking out and building MyTardis from a designated branch of a designated repository. This can be risky. For a production MyTardis instance:

  • It is prudent to use a stable branch of MyTardis rather than 'master' some other development branch.
  • Consider creating your own MyTardis fork and using that so that you don't get surprise redeployments. (Especially if you are tracking 'master'.)
  • It is prudent to try out redeployments in a Test or UAT instance rather than redeploying straight into production.

Configuration

This doesn't really differ from other Chef cookbook installations, so I recommend you follow their Getting Started Guide http://wiki.opscode.com/display/chef/Fast+Start+Guide which will help you setup a Chef Client Workstation (to control deployments) and a Hosted Chef account (to store cookbooks and pull them down from the server).

This guide will cover two methods. Vagrant (virtual machine on your local computer) and remote instances.

Local Deployment

Chef-Solo is a very quick and easy way to install MyTardis on a clean VM such as the NeCTAR cloud. It is as simple as pasting a few commands into a terminal. Recommended for first-timers, evaluating MyTardis, and developers setting up a local VM using Vagrant.

See https://github.com/mytardis/mytardis-chef/wiki/Chef-Solo-Guide .

Remote Deployment

This is a guide to deploying this on a remote server. This has been tested using the NecTAR cloud (like Amazon ec2) on Ubuntu 10, 12 and CentOS 6.

This part needs to be done if using Hosted chef (for Chef-Solo help go elsewhere, or if you don't know what this means then use Hosted Chef!).

Follow the Chef getting started guide linked above, then:

  1. Upload the mytardis-chef cookbook to the server
  2. Create knife roles for MyTardis

Upload

On your client workstation (the one with Knife installed), use:

knife cookbook upload -o /path/to/mytardis-chef/site-cookbooks/:/path/to/mytardis-chef/cookbooks/ -a -d

Where /path/to is the path to your downloaded mytardis-chef repo (this repository).

This will upload the MyTardis recipe and its dependencies to your Hosted Chef.

Create Knife Roles

knife role from file /path/to/mytardis-chef/roles/mytardis.json

Your Hosted Chef now has the cookbook, and roles and therefore all it needs to be run on an instance.

Run these commands from your knife client workstation (your local machine). You may need to copy your knife.rb username.pem and client-validator.pem to ~/.chef so you can run the knife command from any directory.

For CentOS 6, where is the ip address of the target server and root is the name of your superuser account. Note: depending on your remote machine, you may need to use key authentication by inserting '-i /path/to/yourkey.pem' in the account.

knife bootstrap <ip> -x root -r 'role[mytardis]'

For Ubuntu 10 (Lucid)

Note: you may have to apt-get update the server first. This is simple by running:

knife bootstrap <ip> -x ubuntu --sudo -r 'recipe[apt]'

knife bootstrap <ip> -x ubuntu --sudo -r 'role[mytardis]'

That's it! You should watch knife invoke chef on the target server and watch the deployment unfold before your eyes.

Amazon EC2 Deployment

Instructions for remotely deploying MyTardis on an Amazon EC2 instance.

Prerequisites:

  • Have a chef knife workstation set up sufficiently so you can communicate with either a Chef Server or Hosted Chef. An example knife.rb is as follows:
log_level                :info
log_location             STDOUT
# your chef-server or hosted chef username
node_name                'admin'
# your key downloaded from chef-server or hosted chef
client_key               '/root/.chef/admin.pem'
# your validator key name downloaded from chef-server or hosted chef
validation_client_name   'chef-validator' 
# your validator key downloaded from chef-server or hosted chef
validation_key           '/root/.chef/validation.pem'
# if using chef server then the URL
chef_server_url          'https://spetznatz.ath.cx'
# usually a subdir of your username on the knife workstation
syntax_check_cache_path  '/root/.chef/syntax_check_cache'
  • An Amazon EC2 account (know your access key ID and secret key)
  • SSH keys created and downloaded for logging into an EC2 instance

Install the knife ec2 plugin:

gem install knife-ec2 (or see https://github.com/opscode/knife-ec2)

Here's a sample command that spins up an EC2 instance of CentOS 6.2 and deploys MyTardis.

knife ec2 server create \ 
    -r "role[mytardis-3]" \
    -I ami-8035b9b0 --flavor m1.small \
    -x ec2-user -N mytardis \
    --aws-access-key-id "FROM_AMAZON_EC2_CONSOLE" \
    --aws-secret-access-key "FROM_AMAZON_EC2_CONSOLE" \
    --ssh-key "CREATED_IN_AMAZON" --identity-file "CREATED_IN_AMAZON.pem" \
    --region us-west-2 --json-attributes "`<solo/node.json`"
  • This cookbook must be uploaded to your chef server or hosted chef (see previous Remote Deployment section)
  • The roles/mytardis.json must be loaded into the server
  • The AMI used is available on the us-west2 (oregon) region. If you decide to deploy elsewhere, then find a new image.
  • The json-attributes is loaded from solo/node.json in this repository as an example. You can create your own if you wish, or edit this one
  • Make sure you've set up default security rules on the EC2 management console to allow connections to port 80, or else you'll end up with a server you can't connect to!
  • Connecting to your server's IP in the browser doesn't work, but connecting to the hostname (FQDN) does, so beware.

mytardis-chef's People

Contributors

adamhjk avatar crawley avatar grischa avatar mattray avatar stevage avatar steveandroulakis avatar tjdett avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mytardis-chef's Issues

Django 1.5 MyTardis fails to start

Django 1.5 requires settings.py to have a variable, ALLOWED_HOSTS set with the hostname of the server deployed. This means that by default this chef installation doesn't start.

Write code to insert a hostname or ip in the startup script settings.py

The mytardis chef cookbook license says "all rights reserved"

I'm pretty sure that this is an oversight (going back to the Tim's initial version!) but the "license" in site-cookbooks/mytardis/metadata.rb is specified as "All rights reserved".

Can we fix this? I recommend using a "BSD" license ...

(I'm about to refactor the cookbook for the "nectar-cookbooks" project, and I just want to make sure that the "i's are dotted and t's are crossed".)

Example data loading needs to be switchable / configurable and compatible with master

This dodgy code loads a few sample experiments with some metadata, a test account and some schemas (including licenses):

https://github.com/mytardis/mytardis-chef/blob/master/site-cookbooks/mytardis/recipes/default.rb#L140

It worked for mytardis/3.0 but won't work for master given the db has changed.

It needs a few things:

  1. Example data should be switchable (on and off) via chef node or role configuration
  2. The nature of the example data should be configurable (ie give me only experiments from this source please, or of type this, with these test accounts)
  3. Currently the configuration is hard-coded into json fixtures and a tar for the files extracted into the store. This needs to be much smarter eg grab and load METS experiments and download files from a remote source using the existing remote file-grabbing architecture in Mytardis. Just a suggestion.

I caused this mess to get a result for 3.0, so I'll take care of fixing it and making it more sophisticated (with input on how to do this appreciated).

The mytardis-chef recipes shouldn't mess with firewall rules

SEE mytardis/mytardis#132 for discussion.

I discovered this issue with my own in-house fork of the mytardis chef cookbook. Basically, I'm using the recipes in the context of a Chef installation for a large number of our servers, and (naturally) we want the firewall settings to be configurable according to the specific requirements of each node.

When I tried integrating our local firewall stuff into a machine using my mytardis cookbooks, I discovered that the nginx.rb recipe is unilaterally updating the firewall rules to allow universal SSH, HTTP and HTTPS access. Now this might be OK on a cloud virtual running a non-production MyTardis instance. But it is NOT OK on a production MyTardis system. And it is certainly not OK that the nginx.rb recipe is messing with the SSH access settings.

The fix is simple ... just remove the last 3 lines of the recipe. But it might cause you to need to make other changes elsewhere to compensate (in the cloud use-case) so I'm hesitant to submit a pull request.

The real problem here is that the Chef various iptables recipes (all of them I've been able to find!) work on the basis that they are creating an "iptables" file from scratch, rather than updating existing firewall settings. So basically if the "nginx.rb" file is going to create rules to allow HTTP and HTTPS, then >>something<< has to be creating a rule for SSH ... or else Chef is going to "brick" SSH access. Somebody took a pragmatic decision that "nginx.rb" should add the SSH rule, but that has bad consequences. In fact, the better solution is for the core mytardis recipes to NOT mess with this.

Fail on Ubuntu 13.10 due to lack of Curl

The Chef Solo guide for installing MyTardis on Ubuntu fails. This is due to the assumption of the instruction about the availability of the curl command. However, a NeCTAR VM that is created using the NeCTAR Ubuntu 13.10 image does not have curl. So, the instruction should be updated to ask users to install curl.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.