XDump is an utility to make partial consistent dump and load it into the database.
The idea is to provide an ability to specify what to include in the dump via SQL queries.
XDump can be obtained with pip
:
$ pip install xdump
Making a dump (on production replica for example):
>>> from xdump.postgresql import PostgreSQLBackend
>>>
>>> backend = PostgreSQLBackend(dbname='app_db', user='prod', password='pass', host='127.0.0.1', port='5432')
>>> backend.dump(
'/path/to/dump.zip',
full_tables=['groups'],
partial_tables={'employees': 'SELECT * FROM employees ORDER BY id DESC LIMIT 2'}
)
Load a dump on you local machine:
>>> backend = PostgreSQLBackend(dbname='app_db', user='local', password='pass', host='127.0.0.1', port='5432')
>>> backend.load('/path/to/dump.zip')
You don't have to specify all queries for related objects - XDump will load them for you automatically. It covers
both, recursive and non-recursive relations.
For example, if the employees
table has foreign keys group_id
(to groups
table) and manager_id
(to employees
table) the resulting dump will have all objects related to selected employees
(as well as for objects related to related objects, recursively).
At the moment only the following are supported:
- PostgreSQL
- SQLite
Add xdump.extra.django
to your INSTALLED_APPS
settings:
INSTALLED_APPS = [
...,
'xdump.extra.django',
]
Add XDUMP
to your project settings file. It should contain minimum 2 entries:
- FULL_TABLES - a list of tables, that should be fully dumped.
- PARTIAL_TABLES - a dictionary with
table_name
:select SQL
XDUMP = {
'FULL_TABLES': ['groups'],
'PARTIAL_TABLES': {'employees': 'SELECT * FROM employees WHERE id > 100'}
}
Optionally you could use a custom backend:
XDUMP = {
...,
'BACKEND': 'importable.string',
}
Run xdump
command:
$ ./manage.py xdump dump.zip
Run xload
command:
$ ./manage.py xload dump.zip
Possible options to both commands:
alias
- allows you to choose database config from DATABASES, that is used during the execution;backend
- importable string, that leads to custom dump backend class.
The following make
command could be useful to get a configured dump from production to your local machine:
sync-production:
ssh -t $(TARGET) "DJANGO_SETTINGS_MODULE=settings.production /path/to/manage.py xdump /tmp/dump.zip"
scp $(TARGET):/tmp/dump.zip ./dump.zip
ssh -t $(TARGET) "rm /tmp/dump.zip"
DJANGO_SETTINGS_MODULE=settings.local $(PYTHON) manage.py xload ./dump.zip
And usage is:
$ make sync-production [email protected] PYTHON=/path/to/python/in/venv
XDump supports Python 3.4, 3.5 & 3.6.