Giter Club home page Giter Club logo

dump's Introduction

Gem Version Build Status Rubocop Code Climate

Dump

Rails app rake and capistrano tasks to create and restore dumps of database and assets.

Tested against rails 2.3, 3.1, 3.2, 4.0, 4.1, 4.2, 5.0, 5.1, 5.2, 6.0, 6.1.

Works with ruby 1.9.3, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, nominally works with jruby (see check workflow).

Install

Put in Gemfile if you are using bundler:

gem 'dump'

Install as plugin for rails 3 (not recommended):

rails plugin install git://github.com/toy/dump.git

Install as plugin for rails 2:

script/plugin install git://github.com/toy/dump.git

Capistrano integration

To get capistrano tasks in rails 3, put in config/deploy.rb:

require 'dump/capistrano'

Assets config

config/assets holds paths of dirs you want to dump in file:

public/audios
public/flash
public/images/upload
public/videos

Generate in rails 3:

rails generate assets_config

Generate in rails 2:

script/generate assets_config

Capistrano integration

When using cap tasks, dump folder should be in persistent place and be linked to application folder, or you will lose all dumps every deploy. Default recipe creates link on after deploy:update_code.

You can use cap dump:* tasks to control dumps on remote server. Don't forget to deploy application to remote server before using dump:remote tasks. Also you can set custom remote rake binary in your deploy.rb like:

set :rake, "/custom/rake"

Usage

# create dump
rake dump
rake dump:create

# list avaliable dumps
rake dump:versions

# restore dump
rake dump:restore

# delete old and unfinished dumps (all non tgz files will be deleted if they are not locked)
rake dump:cleanup

Environment variables

While creating dumps:

DESC, DESCRIPTION — free form description of dump

rake dump DESC='uploaded photos'

TAGS, TAG — comma separated list of tags

rake dump TAGS='photos,videos'

ASSETS — comma or colon separated list of paths or globs to dump

rake dump ASSETS='public/system:public/images/masks/*'
rake dump ASSETS='public/system,public/images/masks/*'

TABLES — comma separated list of tables to dump or if prefixed by "-" — to skip; by default only sessions table is skipped; schema_info and schema_migrations are always included if they are present

dump all tables except sessions:

rake dump

dump all tables:

rake dump TABLES='-'

dump only people, pages and photos tables:

rake dump TABLES='people,pages,photos'

dump all tables except people and pages:

rake dump TABLES='-people,pages'

While restoring dumps:

LIKE, VER, VERSION — filter dumps by full dump name

rake dump:versions LIKE='2009'
rake dump:restore LIKE='2009' # restores last dump matching 2009

TAGS, TAG — comma separated list of tags without '+' or '-' — dump should have any of such tags prefixed with '+' — dump should have every tag with prefix prefixed with '-' — dump should not have any of tags with prefix

select dumps with tags photos or videos:

rake dump:restore TAGS='photos,videos'

select dumps with tags photos and videos:

rake dump:restore TAGS='+photos,+videos'

skip dumps with tags mirror and archive:

rake dump:restore TAGS='-mirror,-archive'

select dumps with tags photos or videos, with tag important and without mirror:

rake dump:restore TAGS='photos,videos,+important,-mirror'

MIGRATE_DOWN — don't run down for migrations not present in dump if you pass "0", "no" or "false"; pass "reset" to recreate (drop and create) db by default all migrations which are not present in dump are ran down

don't run down for migrations:

rake dump:restore MIGRATE_DOWN=no

reset database:

rake dump:restore MIGRATE_DOWN=reset

RESTORE_SCHEMA — don't read/change schema if you pass "0", "no" or "false" (useful to just restore data for table; note that schema info tables are also not restored)

don't restore schema:

rake dump:restore RESTORE_SCHEMA=no

RESTORE_TABLES — works as TABLES, but for restoring

restores only people, pages and photos tables:

rake dump RESTORE_TABLES='people,pages,photos'

restores all tables except people and pages:

rake dump TABLES='-people,pages'

RESTORE_ASSETS — works as ASSETS, but for restoring

rake dump RESTORE_ASSETS='public/system/a,public/system/b'
rake dump RESTORE_ASSETS='public/system/a:public/images/masks/*/new*'

For listing dumps:

LIKE, VER, VERSION and TAG, TAGS as for restoring

SUMMARY — output info about dump: "1", "true" or "yes" for basic info, "2" or "schema" to display schema as well

rake dump:versions SUMMARY=1
rake dump:versions SUMMARY=full # output schema too

For cleanup:

LIKE, VER, VERSION and TAG, TAGS as for restoring

LEAVE — number of dumps to leave

rake dump:cleanup LEAVE=10
rake dump:cleanup LEAVE=none

cap tasks

For all cap commands environment variables are same as for rake tasks

TRANSFER_VIA — transfer method (rsync, sftp or scp) By default transferring task will try to transfer using rsync if it is present, else it will try to use sftp and scp

force transfer using scp:

cap dump:remote:download TRANSFER_VIA=scp
cap dump:mirror:down TRANSFER_VIA=scp

BACKUP, AUTOBACKUP, AUTO_BACKUP — no autobackup if you pass "0", "no" or "false" by default backup is created before mirroring

don't create backup:

cap dump:mirror:down BACKUP=0
cap dump:mirror:down AUTOBACKUP=no
cap dump:mirror:down AUTO_BACKUP=false

Basic cap tasks are same as rake tasks

cap dump:local
cap dump:local:create
cap dump:local:restore
cap dump:local:versions
cap dump:local:cleanup

cap dump:remote
cap dump:remote:create
cap dump:remote:restore
cap dump:remote:versions
cap dump:remote:cleanup

Dump exchanging tasks

transfer selected dump to remote server:

cap dump:local:upload

transfer selected dump to local:

cap dump:remote:download

Mirroring tasks

mirror local to remote (create local dump, upload it to remote and restore it there):

cap dump:mirror:up

mirror remote to local (create remote dump, download it from remote and restore on local):

cap dump:mirror:down

Backuping tasks

backup remote on local (create remote dump and download it):

cap dump:backup:create

restore backup (upload dump and restore on remote):

cap dump:backup:restore

If you have separate db server

by default, dump's capistrano tasks use remote server with role :db, :primary => true. If you have separate db server, and want to use dump with assets, you should run cap tasks as following:

cap dump:mirror:down HOSTS=myappserver.com

where myappserver.com is application server carrying assets with database access.

Copyright

Copyright (c) 2008-2019 Ivan Kuchin. See LICENSE.txt for details.

dump's People

Contributors

aishek avatar be9 avatar esteil avatar igorgladkoborodov avatar irfanpirbhai avatar toy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dump's Issues

Recieve message: 'Capistrano 3 is not yet supported'

I'm following this for rails 3 section in order to include cap tasks. Then I run bin/cap staging dump:remote:create, I recieve message:

(Backtrace restricted to imported tasks)
cap aborted!
Capistrano 3 is not yet supported

Tasks: TOP => staging
(See full trace by running task with --trace)

Environment: rails 4.2.0, ruby 2.0.0p598, capistrano 3.3.5, dump 1.1.0

Is this tasks are not supposed to use with rails 4.2?

Rails 2.3.8 deprecation warning

DEPRECATION WARNING: Rake tasks in vendor/plugins/dump/tasks and vendor/plugins/dump/tasks are deprecated. Use lib/tasks instead.

Recieve message: 'invalid input syntax for type inet'

I have devise based users table with couple inet fields.

  create_table "users", force: :cascade do |t|
    ...
    t.datetime "last_sign_in_at"
    t.inet     "current_sign_in_ip"
    t.inet     "last_sign_in_ip"
    t.string   "confirmation_token"
    ...
  end

Then I try to restore database I recieve error:

ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR:  invalid input syntax for type inet: "--- !ruby/object:IPAddr
family: 2
addr: 1539998154
mask_addr: 4294967295
"

Environment: rails 4.2.0, ruby 2.0.0p598, dump 1.1.0, PostgreSQL 9.3.6

Switch from using Marshal

Oj? Writing/reading object dumps consecutively won't work, but writing/reading lines will.
Allow reading marshalled dumps and add option to select serialisation engine (for example if jruby will be used).

cap dump:backup:create failed

cap dump:backup:create
 * executing `dump:backup:create'
 * executing `dump:remote:create'
 * executing "cd /home/user/application/current; bundle\\ exec\\ rake -s dump:create PROGRESS_TTY\\=\\+ RAILS_ENV\\=production TAGS\\=remote,backup"
  servers: ["server"]
  [server] executing command
  command finished in 461ms
failed: "rvm_path=/usr/local/rvm /usr/local/rvm/bin/rvm-shell 'ruby-1.9.3-p194' -c 'cd /home/user/application/current; bundle\\ exec\\ rake -s dump:create PROGRESS_TTY\\=\\+ RAILS_ENV\\=production TAGS\\=remote,backup'" on server

dump_command shelljoin?

add special role to capistrano recipies to dump from

In my recent project I have separate app server and db server, so assets and db access granted only from app. But for now I cannot use gem dump for my project, because it use only db as role to dump.

I think that adding special role for this purpose will be fine.

Dump file path

Is there any way to specify the dump path? Currently it is in "dump/" . I want to change the path.

Mediumtext transforms to Longtext

I've got mysql database. One table has column with type mediumtext (16Mb).
So I create dump, restore it, .. And column transforms to longtext (2Gb)!

$ rake db:drop && rake db:create && rake db:migrate
$ ack frames db/schema.rb
    t.text     "frames",     :limit => 16777215
$ rake dump:create && rake dump:restore
$ ack frames db/schema.rb
    t.text     "frames",     :limit => 2147483647

I've tried to find out what happens, but couldn't. Can you?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.