Giter Club home page Giter Club logo

pgreplicationtutorial's Introduction

pgReplicationTutorial

This repository contains files for the PostgreSQL Binary Replication tutorial.

These files are required to perform the hands-on exercises. Importantly, there is significant setup required in order to do the hands-on exercises which needs to happen before the tutorial starts, so please read the below.

All steps through Vagrant Up / Docker Pull should be completed before getting to the tutorial. The tutorial will not include any help on getting these systems running.

Requirements

Laptop Harware:

  • running Linux, OSX, or Windows XP or later
  • 64-bit (32-bit may be possible, see below)
  • at least 1GB of RAM, 2GB or more preferred
  • 300MB free disk space

Software and Wetware:

  • terminal program capable of ssh
  • familiarity with the bash/linux command line
  • familiarity with one or more command-line text editors
  • vagrant and virtualbox (see below) OR Docker 1.5.0 or later

Vagrant vs. Docker

Docker is the recommended platform for running the execises, because it is much easier to set up, and requires far less system resources to run when you're taking the tutorial. However, using Docker does require:

  • A Linux laptop, with Linux kernel 3.8 or later
  • Package lxc-docker installed running version 1.5 or later

OR:

  • A windows or Mac laptop running the Docker Toolbox or Kitematic

Docker Toolbox is available from Docker. We are not able to provide detailed instructions or assistance on configuring Docker Toolbox on Windows or Mac. The Toolbox page, however, has an excellent tutorial.

If you are unable to get Docker working, or are simply more comfortable with Vagrant, then please see the Vagrant instructions in Vagrant.md after you have installed the tutorial exercises.

Installing Tutorial Exercises

Install the tutorial exercises on your machine one of two ways below. Installing them is required if you are using Vagrant, and is optional if you are using Docker, but helpful.

Preferred Method: Git Checkout from the Github repo. The repository is here: https://github.com/jberkus/pgReplicationTutorial, and you can clone it by:

git clone https://github.com/jberkus/pgReplicationTutorial.git

Alternate Method: if you're not comfortable with git, download the tarball from:

https://dl.dropboxusercontent.com/u/5132935/pgReplicationTutorial.tgz

This will require the programs "tar" and "gzip" to expand, as follows:

tar -p -xvf pgReplicationTutorial.tgz

There is also a "zip" formatted archive which should be more friendly for Windows users. However, this zip file will not work for Vagrant execution, so it is only suitable for Docker users. You can obtain this zip file from github:

https://github.com/jberkus/pgReplicationTutorial/archive/master.zip

The ReplicationTutorial directory should be placed somewhere you have disk space available.

Docker Installation

The Docker image for this tutorial is 64-bit and was built with Docker 1.7.1. If you are on a 32-bit machine, or if you are limited to using Docker 1.4 or older, I recommend that you scroll back up and follow the Vagrant install instructions instead.

If you're working on Windows or OSX, you'll need to run commands using the Docker Terminal which is a part of the Docker Toolbox.

First pull the image:

docker pull jberkus/pgreplicationtutorial

While a pull is sufficient, I suggest that you test the image by doing:

docker run -it --rm jberkus/pgreplicationtutorial

This should log you into a command prompt as root. If you type "exit" the container will halt and erase itself.

Other Files In This Package

The pgReplicationTutorial package also contains the following files in the Tutorial directory:

exercises.txt

This is a set of commands which you will be running during the hands-on exercises for the tutorial. These are provided in a text file so that you can copy & paste, instead of trying to read every letter and dash from the instructor's screen.

pgReplicationTutorial.odt/pdf

These are copies of the slides for the tutorial.

ChangeLog

0.6 Updated for Postgres Open 2015. PostGIS removed to shrink image. Docker instructions provided as primary recommended option.

0.5.1 multiple fixes to Docker image.

0.5 Added PostgreSQL 9.4 replication slots to the tutorial as exercise. Added some additional tuning and configuration portions.

0.4 Version produced for LinuxConf.AU 2015. Removes PostGIS exercises. Upgrades to PostgreSQL 9.4. Adds Docker support.

0.3 Version produced for FOSS4G 2014. Includes information about replicating PostGIS. Interactive "play" demonstrations have have been removed due to their time-consuming nature.

0.2 Initial version produced for DjangoCon 2013.

License

The pgReplicationTutorial is Copyright 2013-2015 Josh Berkus and PostgreSQL Experts Inc.

All slides, text, instructions and similar content in this tutorial are licensed [Creative Commons Attribution-ShareAlike 3.0] (http://creativecommons.org/licenses/by-sa/3.0/us/)

Code exercises, virtual machines, containers and sample databases are licensed under the [Gnu Public License Version 2] (http://www.gnu.org/licenses/gpl-2.0.html)

All other rights reserved.

pgreplicationtutorial's People

Contributors

jberkus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pgreplicationtutorial's Issues

./dev_patroni_cluster.sh - one of the replicas dies during promotion

When creating a cluster of 3 or 5 members, and then stopping the master, it seems that one of the members fails to rejoin the new master; and instead dies with a Python error:

https://gist.github.com/drnic/b6e4fbd9940d8694dded

Thoughts on what might be going on? My Python isn't strong; so can't easily spot what data might be flowing down into etcd.py that causes AttributeError: 'NoneType' object has no attribute 'strip'

This issue did not happen when we then stopped the new master.

That is, when we had 5 members, if we stop the master, we lose the master and one of the replicas and the cluster size is 3. When we stop the new master, we only lose the master, and the cluster size is 2.

When we start with a cluster of 3 members, if we stop the master we lose the master and one of the replicas; and the final cluster size is 1.

What if...

...someone would try to do a sleak and compliant postgres query loadbalancer in lua for haproxy?

https://www.arpalert.org/haproxy-lua.html

I mean is this not the natural thing to do to replace bgbouncer / pgpool and all the other tools?

Nice side effect: haproxy happens to be the lb of choice of many orchestration frameworks under the hood, like rancher and I believe docker swarm and kubernetes also work atop of haproxy or can be configured that way...

Not sure if this is insane or just nobody has asked this question, so far...

Vagrant setup issues

Hi,

When I do vagrant up and connect as a postgres user.
I see only main in 9.4 folder.
I don't see master, replica1 and replica2 folders.
And when I want to do pg_basebackup I get this error:
pg_basebackup: could not connect to server: FATAL: no pg_hba.conf entry for replication connection from host "[local]", user "postgres", SSL off

Resolving permission issues when mounting shared folder from a Windows host.

The synched_folder parameters may resolve the shared folder issue when running the Vagrant boxes under Windows.

puppet.vm.synced_folder 'etc-puppet', '/etc/puppet',
      owner:  'root',
      group:  'root',
      create: true,
      mount_options: ['dmode=775','fmode=664']

These are faked and can't be modified within the VM, and still won't resolve any issues with linked files.

(I've tried it and it helps, but not resolved, may need a different user or group.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.