spilin / capistrano3-postgres Goto Github PK
View Code? Open in Web Editor NEWCreate postgres dumps, download and replicate locally.
License: MIT License
capistrano3-postgres's People
Contributors
Stargazers
Watchers
Forkers
joshuapaling ryanmacg hoppinger dwa012 nbulaj peppster tinynumbers ahvahsky2008 ypadlyak danielschwab pariv vsuhachevcapistrano3-postgres's Issues
Error when running rails s - `undefined method `namespace' `
When running rails s, I was getting:
/lib/capistrano3/tasks/postgres.rb:1:in<top (required)>': undefined method namespace' for main:Object (NoMethodError)
I fixed it by using gem 'capistrano3-postgres', require: false in my Gemfile (ie, require: false is the solution). Just adding this for future people with the same problem.
Set up local folder for backup
cap postgres:backup:download # Downloads dump to local server(By default stores file in ./tmp/postgres_backup directory)
How could i change default folder for local backups? Thanks )
Support for Rails.application.credentials in database.yml
My database.yml file includes something like this:
production:
<<: *default
database: prod
host: localhost
username: <%= Rails.application.credentials.dig :postgres, :username %>
password: <%= Rails.application.credentials.dig :postgres, :password %>Currently, this gem fails at task cap production postgres:backup:create with:
00:00 postgres:backup:ensure_remote_dirs
01 mkdir -p my_path/shared/postgres_backup
✔ 01 my_user@my_path 0.059s
#<Thread:0x00007ff89c960928@/Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from /Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as my_user@my_path: bundle exit status: 1 (SSHKit::Runner::ExecuteError)
bundle stdout: Nothing written
bundle stderr: (erb):28:in `<main>': uninitialized constant Rails (NameError)
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `eval'
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `result'
from -e:16:in `<main>'
#<Thread:0x00007ff89c16d108@/Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:10 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
1: from /Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
/Users/me/.rvm/gems/ruby-2.5.1/gems/sshkit-1.18.0/lib/sshkit/runners/parallel.rb:15:in `rescue in block (2 levels) in execute': Exception while executing as my_user@:my_path Exception while executing as my_user@my_path: bundle exit status: 1 (SSHKit::Runner::ExecuteError)
bundle stdout: Nothing written
bundle stderr: (erb):28:in `<main>': uninitialized constant Rails (NameError)
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `eval'
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `result'
from -e:16:in `<main>'
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as my_user@my_path: Exception while executing as my_user@my_path: bundle exit status: 1
bundle stdout: Nothing written
bundle stderr: (erb):28:in `<main>': uninitialized constant Rails (NameError)
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `eval'
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `result'
from -e:16:in `<main>'
Caused by:
SSHKit::Runner::ExecuteError: Exception while executing as my_user@my_path: bundle exit status: 1
bundle stdout: Nothing written
bundle stderr: (erb):28:in `<main>': uninitialized constant Rails (NameError)
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `eval'
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `result'
from -e:16:in `<main>'
Caused by:
SSHKit::Command::Failed: bundle exit status: 1
bundle stdout: Nothing written
bundle stderr: (erb):28:in `<main>': uninitialized constant Rails (NameError)
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `eval'
from /usr/share/rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/erb.rb:876:in `result'
from -e:16:in `<main>'
Tasks: TOP => postgres:backup:create
(See full trace by running task with --trace)I believe the key error here is: uninitialized constant Rails (NameError). When I inspect this gem's lib/capistrano3/tasks/postgres.rb file, line 162 is the culprit:
# Grabs remote database config before creating dump <---- line 152
def grab_remote_database_config
return if fetch(:postgres_remote_database_config)
on roles(fetch(:postgres_role)) do |role|
within release_path do
env = fetch(:postgres_env).to_s.downcase
filename = "#{deploy_to}/current/config/database.yml"
eval_yaml_with_erb = <<-RUBY.strip
#{env_variables_loader_code(env)}
require 'erb'
puts ERB.new(File.read('#{filename}')).result <------- line 162
RUBY
capture_config_cmd = "ruby -e \"#{eval_yaml_with_erb}\""
yaml_content = test('ruby -v') ? capture(capture_config_cmd) : capture(:bundle, :exec, capture_config_cmd)
set :postgres_remote_database_config, database_config_defaults.merge(YAML::load(yaml_content)[env])
end
end
end <------- line 169Because the scope of line 162 has no way to interpret a call to Rails, my database.yml causes this code to fail.
One obvious, easy work around is to use one of the environment gems that this gem supports. Another solution is to update this gem to support calls to Rails -- something like:
cmd = "cd #{current_path} && /home/my_user/.rvm/gems/ruby-2.5.1/wrappers/rails runner -e production 'puts ERB.new(File.read(\"#{filename}\")).result'"
yaml_content = capture(cmd)
loaded = YAML::load(yaml_content)I am not proficient enough with Capistrano's SSHKit DSL to have figured out a cleaner way to call rails runner, but this is nearly exactly my development code at the moment and its working. A similar change would have to be made to def grab_local_database_config
What are your thoughts on whether this gem should support calls to Rails.application.credentials in the database.yml file?
Fail when using dotenv-rails gem
Hi. cap production postgres:backup:create doesn't work when project uses dotenv gem for working with ENV variables & configs. Log (real names hidden):
00:02 postgres:backup:create
01 PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB
01 pg_dump: [archiver (db)] connection to database "DB" failed: could not translate host name "-Fc" to address: Name or service not known
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as [email protected]: PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB exit status: 1
PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB stdout: Nothing written
PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB stderr: pg_dump: [archiver (db)] connection to database "DB" failed: could not translate host name "-Fc" to address: Name or service not known
SSHKit::Command::Failed: PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB exit status: 1
PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB stdout: Nothing written
PGPASSWORD= pg_dump -h -Fc --file=/var/www/project_name/shared/postgres_backup/db_backup.2016-06-23_18-22-00.sqlc -Z 6 DB stderr: pg_dump: [archiver (db)] connection to database "DB" failed: could not translate host name "-Fc" to address: Name or service not knownGemfile.lock
capistrano-dotenv-tasks (0.1.3)
capistrano (~> 3.0)
dotenv (2.1.1)
dotenv-rails (2.1.1)
dotenv (= 2.1.1)
railties (>= 4.0, < 5.1)database.yml
production:
<<: *default
database: DB
host: <%= ENV['DATABASE_HOST'] %>
username: <%= ENV['DATABASE_USERNAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>As you can see, ENV variables was not setuped. I think we need to manually load Dotenv to properly handle connection configuration. Maybe this gem can be helpful - https://github.com/glyph-fr/capistrano-dotenv-tasks
How override download path?
Hi. I want download backup from vds to my pc, but lib work with unix directory. On my pc - Windows 7. How override download path?
Fails to when reading dotenv files
Runniing cap staging postgres:backup:create to try this it generates a syntax error for Dotenv.
Gemfile:
group :development do
# Deploy
gem 'capistrano', '~> 3.2.0'
gem 'capistrano-bundler', '~> 1.1.2'
gem 'capistrano-rails', '~> 1.1.1'
gem 'capistrano-rvm'
gem 'capistrano-sidekiq', '0.5.3'
gem 'capistrano3-postgres', '0.2.1', require: false
end
Running /usr/bin/env ruby -e "begin; require 'dotenv'; Dotenv.load(; File.expand_path('.env.production'),; File.expand_path('.env'); );
DEBUG [645fcf2e] -e:1: syntax error, unexpected ';', expecting ')'
...require 'dotenv'; Dotenv.load(; File.expand_path('.env.produ...
Output:
DEBUG [40cc0f23] Running /usr/bin/env [ -d ~/.rvm ] as [email protected]
DEBUG [40cc0f23] Command: [ -d ~/.rvm ]
DEBUG [40cc0f23] Finished in 0.823 seconds with exit status 0 (successful).
DEBUG [27643fed] Running ~/.rvm/bin/rvm version as [email protected]
DEBUG [27643fed] Command: ~/.rvm/bin/rvm version
DEBUG [27643fed] rvm 1.25.28 (stable) by Wayne E. Seguin <[email protected]>, Michal Papis <[email protected]> [https://rvm.io/]
DEBUG [27643fed] Finished in 0.513 seconds with exit status 0 (successful).
rvm 1.25.28 (stable) by Wayne E. Seguin <[email protected]>, Michal Papis <[email protected]> [https://rvm.io/]
DEBUG [f5ff06be] Running ~/.rvm/bin/rvm current as [email protected]
DEBUG [f5ff06be] Command: ~/.rvm/bin/rvm current
DEBUG [f5ff06be] ruby-2.1.2
DEBUG [f5ff06be] Finished in 0.550 seconds with exit status 0 (successful).
ruby-2.1.2
DEBUG [fd3bf8f1] Running ~/.rvm/bin/rvm default do ruby --version as [email protected]
DEBUG [fd3bf8f1] Command: ~/.rvm/bin/rvm default do ruby --version
DEBUG [fd3bf8f1] ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
DEBUG [fd3bf8f1] Finished in 0.935 seconds with exit status 0 (successful).
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
INFO [7729821e] Running /usr/bin/env mkdir -p /var/www/myapp/shared/postgres_backup as [email protected]
DEBUG [7729821e] Command: /usr/bin/env mkdir -p /var/www/myapp/shared/postgres_backup
INFO [7729821e] Finished in 0.106 seconds with exit status 0 (successful).
DEBUG [5d64990d] Running /usr/bin/env if test ! -d /var/www/myapp/current; then echo "Directory does not exist '/var/www/myapp/current'" 1>&2; false; fi as [email protected]
DEBUG [5d64990d] Command: if test ! -d /var/www/myapp/current; then echo "Directory does not exist '/var/www/myapp/current'" 1>&2; false; fi
DEBUG [5d64990d] Finished in 0.100 seconds with exit status 0 (successful).
DEBUG [9947b051] Running /usr/bin/env ruby -v as [email protected]
DEBUG [9947b051] Command: ruby -v
DEBUG [9947b051] ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
DEBUG [9947b051] Finished in 0.097 seconds with exit status 0 (successful).
DEBUG [645fcf2e] Running /usr/bin/env ruby -e "begin; require 'dotenv'; Dotenv.load(; File.expand_path('.env.production'),; File.expand_path('.env'); ); rescue LoadError; end; require 'erb'; puts ERB.new(File.read('/var/www/myapp/current/config/database.yml')).result" as [email protected]
DEBUG [645fcf2e] Command: ruby -e "begin; require 'dotenv'; Dotenv.load(; File.expand_path('.env.production'),; File.expand_path('.env'); ); rescue LoadError; end; require 'erb'; puts ERB.new(File.read('/var/www/myapp/current/config/database.yml')).result"
DEBUG [645fcf2e] -e:1: syntax error, unexpected ';', expecting ')'
...require 'dotenv'; Dotenv.load(; File.expand_path('.env.produ...
... ^
Gem is outdated
Hi, the published gem doesn't have the latest updates to fix the defaults issue meaning I can't use the rubygems version in production. Is there any plan to do a release?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.