famoseagle / skytools-dev Goto Github PK

= SkyTools - tools for PostgreSQL =

This is a package of tools in use in Skype for replication and
failover.  Also it includes a generic queuing mechanism PgQ and
utility library for Python scripts.

== Overview ==

It contains following tools:

=== PgQ ===

This is the queue mechanism we use. It consists of PL/pgsql, and C code
in database, with Python framework on top of it.  It is based on
snapshot based event handling ideas from Slony-I, written for general
usage.

Features:

- There can be several queues in database.
- There can be several producers than can insert into any queue.
- There can be several consumers on one queue and all consumers see
   all events.

Documentation:

- PgQ admin tool (pgqadm) usage: doc/pgq-admin.txt
- PgQ SQL API overview: doc/pgq-sql.txt
- PgQ SQL reference: http://skytools.projects.postgresql.org/pgq/

=== Londiste ===

Replication tool written in Python, using PgQ as event transport.

Features:

- Tables can be added one-by-one into set.
- Initial COPY for one table does not block event replay for other
  tables.
- Can compare tables on both sides.

Documentation:

- Londiste script usage: doc/londiste.cmdline.txt
  (also available as `man 1 londiste`)
- Londiste configuration: doc/londiste.config.txt
  (also available as `man 5 londiste`)
- Londiste reference: doc/londiste.ref.txt

=== walmgr ===

This script will setup WAL archiving, does initial backup and
runtime WAL archive and restore.


== Source tree contents ==

  doc/ 
    Documentation in asciidoc format.  Source for both html
    and man pages.

  python/
    Python modules and primary executables - walmgr, londiste, qadmin, pgqadm.

  python/pgq/
    Python framework for PgQ.

  python/londiste/
    Londiste replication.

  python/skytools/
    Low-level utilities for writing database scripts in Python.

  sql/
    Database modules.

  sql/pgq/
    Table definitions and functions for PgQ queueing.

  sql/pgq_coop/
    Functions for cooperative consuming.

  sql/londiste/
    Table definitions and functions for Londiste replication.

  sql/ticker/
    PgQ ticker written in C.
 
  scripts/
    Python scripts with lesser priority.

  lib/
    libusual C libary, for pgqd.

  debian/
    Debian packaging.  This is for creating private packages,
    official Debian packages uses it's own packagin code.

  misc/
    Random scripts used for building.