This is the control repo for the Land Registry Charges project. It contains all the Infrastructure code necessary to provision our entire infrastructure.
Table of Contents
- Security Considerations
- Pre-requisites
- Usage
- Anatomy of this repo
Currently there are no "secure" assets or configuration, e.g. Private-Public key pairs, account passwords, usernames, etc. in this repo. This is how it should be. Since all code is public this really only means the Private-Public key pairs needed to SSH in are placed by hand on new boxes before puppet provisions the box.
Running a vagrant up
on this repo will allow anyone to provision our full
infrastructure locally. Since all the code for the apps is public, including
the Puppet modules which deploy those apps, we see no need to make the
infrastructure code private.
Since the Public-Private key pairs used to secure SSH in the Integration environment are not included here open sourcing this infrastructure code does not detriment the security of the Integration environment.
A cursory glance through this repo will show that there is very little happening here, mostly just composing already public Puppet code.
We use a few plugins in the Dev VM you should install first:
> vagrant plugin install vagrant-dns
> vagrant plugin install vagrant-librarian-puppet
> vagrant plugin install vagrant-cachier
Then start the vagrant dns plugin:
> vagrant dns --install
You also should have Vagrant and Virtualbox. Easiest way is through homebrew-cask:
Install homebrew-cask:
> brew install caskroom/cask/brew-cask
Then install the Vagrant and Virtualbox:
> brew cask install vagrant
> brew cask install virtualbox
This README assumes you have already read the Dev VMs README, which explains many of the concepts used in this repo.
First clone the repo:
> git clone [email protected]:LandRegistry/dev-vm.git
Then install the dependencies:
> librarian-puppet install
Add an environment variable that points at your development folder (e.g. mine is
in ~/Projects/land-registry/charges
) to your ~/.bashrc
or ~/.zshrc
:
> echo "export development=/path/to/your/code" >> ~/.zshrc
And finally you can start the VM:
> vagrant up
And wait, and wait, and wait.
The Vagrantfile
defines a number of VMs so that you can mimic Integration locally. This allows you
to run a change locally and predict whether it will break Integration.
The list of defined VMs is in the Vagrantfile
.
Each box or VM deployed to by Control is provisioned based on the node it matches
in site.pp
.
Nodes are named and the name is used by puppet to determine which node to use for a VM or box.
node 'deedapi' {
include ::profiles::deedapi
}
In this definition any VMs named deedapi
, deedapi.foo
, deedapi.foo.bar.gov.uk
,
etc will be defined based on that node. We use the profiles concept to disconnect
apps from the nodes themselves, so one box could perform the role of Deed Api and
Scribe if you want.
Profiles are stored in The site
folder.
Profiles are stored in this folder. A profile is just a Puppet class that composes the required classes for a box to perform in a given role. For example you may need the App itself, the Standard Environment (python, gcc, etc), a hardened Nginx and IPTables configuration, etc. So we don't overly bog down the developers we only expect Puppet modules in each App to know how to deploy that App, not fully harden it for production use.
Like in the Dev VM we use Hiera for overriding configuration values. This time though we have multiple files, which store the correct config for each box. The correct datafile is chosen based on the Fully Qualified Domain Name. This FQDN should be identical to the URL of the box, whether that is through external public DNS or internal DNS.
This script, located in bin/jenkins.sh
,
is used by jenkins to run puppet on each box it should deploy on to. It requires
only one variable, $WORKSPACE
, which is the location jenkins has checked out
this repo to on the box.
This script will wire together the different folders needed, install the modules
needed, and then run puppet apply
. Since the configuration of a box is based on
it's FQDN we don't need any change from box to box. Puppet will find the correct
node and hiera file and perform
the installation.