The Backup and Restore SDK BOSH release is used for two distinct things:

1. enabling release authors to incorporate database backup & restore functionality in their releases
2. enabling operators to configure their deployments which use external blobstores to be backed up and restored by BBR

Docs: Release Author Guide

Slack: #bbr on https://slack.cloudfoundry.org

Pivotal Tracker: https://www.pivotaltracker.com/n/projects/1662777

Backup and Restore SDK Release status

This repository using master as the main branch, tested releases are tagged with their versions.

Name	Version	Status
MariaDB	10.1.x
MySQL	5.5.x
MySQL	5.6.x
MySQL	5.7.x
Postgres	9.4.x
Postgres	9.6.x

Release authors wanting to write backup and restore scripts frequently need to back up and restore databases (or parts of databases).

Rather than have every team figure out the vagaries of backing up all the different kinds of database supported by CF, we've done it for you. The Backup and Restore SDK abstracts away the differences between databases, offering a consistent interface for your backup and restore scripts to use.

Behind the scenes, the SDK parses a configuration file passed to it, which selects the appropriate database backup/restore strategy (e.g. pg_dump or mysql at the required version) and places the backup artifact in the specified location.

The SDK accepts a JSON document with the following fields:

• postgres (auto-detects versions between 9.4.x and 9.6.x)
• mysql (auto-detects MariaDB 10.1.x, and MySQL 5.5.x, 5.6.x, 5.7.x. Any other mysql variants are not tested)

Users of cf-deployment can simply apply the backup-restore opsfile. This will deploy the database-backup-restorer job on a backup restore VM alongside Cloud Foundry.

You should co-locate the database-backup-restorer job and your release backup scripts on the same VM. If you use a dedicated backup-and-restore VM instance, co-locate them together on that VM. BOSH Lite is supported for testing.

Example BOSH v2 deployment manifest:

...
instance_groups:
- name: backup
  networks:
  - name: my-network
  persistent_disk_type: 10GB
  stemcell: default
  update:
    serial: true
  vm_type: m3.large
  azs: [z1]
  instances: 1
  jobs:
  - name: backup-scripts
    properties:
      mydb:
      address: mydb.example.com
      db_scheme: mysql
      port: 3306
    release: my_release
  - name: database-backup-restorer
    release: backup-and-restore-sdk
...

Your job should template a config.json as follows:

{
  "username": "db user",
  "password": "db password",
  "host": "db host",
  "port": 3306,
  "adapter": "db adapter; see 'Supported database adapters'",
  "database": "name of database to back up",
}

Or if you want to operate on specific tables:

{
  "username": "db user",
  "password": "db password",
  "host": "db host",
  "port": 3306,
  "adapter": "db adapter; see 'Supported database adapters'",
  "database": "name of database to back up",
  "tables": ["list", "of", "tables", "to", "back", "up"]
}

For the full list of config.json properties see Config options.

An example of templating using BOSH Links can be seen in the cf networking release.

In your release backup script, call database-backup-restorer/bin/backup:

/var/vcap/jobs/database-backup-restorer/bin/backup --config /path/to/config.json --artifact-file $BBR_ARTIFACT_DIRECTORY/artifactFile

In your release restore script, call database-backup-restorer/bin/restore:

/var/vcap/jobs/database-backup-restorer/bin/restore --config /path/to/config.json --artifact-file $BBR_ARTIFACT_DIRECTORY/artifactFile

The restore script will assume that the database schema has already been created, and matches the one of the backup. For BOSH releases, this usually means restore can be called after a successful deploy of the release, at the same version as the backup was taken.

For an example of the SDK being used in a release that can be backed up by BBR see the exemplar release.

BBR only supports the backup and restore of blobstores stored in versioned, Amazon S3 buckets and in S3-compatible buckets that are versioned and support AWS Signature Version 4. For more details about enabling versioning on your blobstore, see the Cloud Foundry documentation.

External blobstores are backed up by storing the current version of each blob, not the actual files. Those versions will be set to be the current versions at restore time. This makes backups and restores faster, but also means that restores only work if the original bucket still exists. For more information, see the Cloud Foundry documentation.

Name	Status
Versioned S3

cf-deployment includes ops files for enabling the backup and restore of external blobstores. See the Cloud Foundry documentation for more details.

The external blobstore backup and restore scripts are contained in the s3-versioned-blobstore-backup-restorer job. Locate the job on any of the instance groups in your deployment that have a volume attached (i.e. the /var/vcap/store folder should exist).

The s3-versioned-blobstore-backup-restorer job can be configured using the following properties:

• enabled [Boolean]: enables the backup and restore scripts. true by default.
• buckets [Hash]: a map from bucket identifiers their their configuration. For each bucket, you'll need to specify the following properties:
  • name [String]: the bucket name
  • region [String]: the bucket region
  • aws_access_key_id [String]: the AWS access key ID for the bucket
  • aws_secret_access_key [String]: the AWS secret access key for the bucket
  • endpoint [String]: the endpoint for your storage server, only needed if you are not using AWS S3