seuros / capistrano-sidekiq Goto Github PK

View Code? Open in Web Editor NEW

556.0 14.0 305.0 269 KB

Sidekiq integration for Capistrano

License: Other

Ruby 80.27% HTML 19.73%

capistrano-sidekiq's Introduction

Capistrano::Sidekiq

Sidekiq integration for Capistrano

Installation

gem 'capistrano-sidekiq', group: :development

And then execute:

$ bundle

Usage

    # Capfile
    require 'capistrano/sidekiq'
    install_plugin Capistrano::Sidekiq  # Default sidekiq tasks
    # Then select your service manager
    install_plugin Capistrano::Sidekiq::Systemd

Configurable options - Please ensure you check your version's branch for the available settings - shown here with defaults:

:sidekiq_roles => :worker
:sidekiq_default_hooks => true
:sidekiq_env => fetch(:rack_env, fetch(:rails_env, fetch(:stage)))
# single config
:sidekiq_config_files, ['sidekiq.yml']
# multiple configs
:sidekiq_config_files, ['sidekiq.yml', 'sidekiq-2.yml'] #  you can also set it per server

Example

A sample application is provided to show how to use this gem at https://github.com/seuros/capistrano-example-app

Configuring the log files on systems with less recent Systemd versions

The template used by this project assumes a recent version of Systemd (v240+, e.g. Ubuntu 20.04).

On systems with a less recent version, the append: functionality is not supported, and the Sidekiq log messages are sent to the syslog.

It's possible to workaround this limitation by configuring the system logger to filter the Sidekiq messages; see wiki.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

capistrano-sidekiq's People

Contributors

Stargazers

Watchers

Forkers

jlecour cntd wonyoungso fred 0x616e676572 mikian mdpatrick jwachira rogercampos xinminlabs joshmyers muhammadn tribble swistaczek elbarto132 dpaluy alexyakubenko saicheg remz99 asf bogda shopperplus donbobka envek brain-geek sammylin ivanovv islue christhekeele zshannon robinboening khodzha raulbrito bencrouse dersascha baweaver maruware apolzon dkarter klacointe yeyong123 geun robinclowers rainlabs rstrobl shukob wemcdonald mightysignal parubets reynardmh chrislusted mcb pedrogarrett ntraum daconsortium silentshade azhao1981 n-studio hbin dclausen mcfish lunalium harvesthq felipejfc blainejohnson17 danoph retreaver sauliusgrigaitis yarbro w1mvy alouanemed jhollinger a-w-k hirocaster tango-io martin91 ricardokdz zocoi manishval threetreeslight phillbaker bobstrange aleksevd larryzhao ealden linkedin-inc haikulearning martinkangur mistidoi dtaniwaki lcpmarvel jasonzyj glebtv anno9010 eumlab fcm-digital maketimeinc skillstream kirrmann vgvinay2

capistrano-sidekiq's Issues

Capistrano 3 sidekiq recipe isn't picking up my parameters

I'm not sure if i'm doing something wrong or what but when I set the sidekiq options in my deploy.rb file nothing gets picked up. It just keeps using the default options.

I'm using

capistrano 3.2.1
capistrano-sidekiq 0.3.3

In my deploy.rb file I have these options set

set :sidekiq_pid, "#{shared_path}/pids/sidekiq.pid"
set :sidekiq_log, "#{shared_path}/log/sidekiq.log"
set :sidekiq_config, "#{current_path}/config/sidekiq.yaml"

But when I run cap production deploy it prints out

INFO[3adebbc4] Running ~/.rbenv/bin/rbenv exec bundle exec sidekiq -d -i 0 -P /apps/my_app/shared/pids/sidekiq.pid -e production -L /apps/my_app/current/log/sidekiq.log on my_app.com
DEBUG[3adebbc4] Command: cd /apps/my_app/current && ( RBENV_ROOT=~/.rbenv RBENV_VERSION=2.1.0 ~/.rbenv/bin/rbenv exec bundle exec sidekiq -d -i 0 -P /apps/my_app/shared/pids/sidekiq.pid -e production -L /apps/my_app/current/log/sidekiq.log )

I'm not sure what i'm doing or not doing for it not to pick on the options?

Setting pty to false by default

Any objection ?

Init support?

Rather than have capistrano start/stop/restart Sidekiq directly, can it integrate with upstart/systemd to control the process via init?

`sidekiq_tag' not found (IndexError)

* 2014-07-25 18:20:52 executing `deploy:restart'
    triggering before callbacks for `deploy:restart'
  * 2014-07-25 18:20:52 executing `sidekiq:restart'
  * 2014-07-25 18:20:52 executing `sidekiq:stop'
  * executing "if [ -d /home/deployer/current ] && [ -f /home/deployer/shared/pids/sidekiq.pid ] && kill -0 `cat /home/deployer/shared/pids/sidekiq.pid`> /dev/null 2>&1; then cd /home/deployer/current && bundle exec sidekiqctl stop /home/deployer/shared/pids/sidekiq.pid 10 ; else echo 'Sidekiq is not running'; fi"
    servers: ["x"]
    [x] executing command
 ** [out :: x] Sidekiq is not running
    command finished in 125ms
  * 2014-07-25 18:20:52 executing `sidekiq:start'
/Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:82:in `block in fetch': `sidekiq_tag' not found (IndexError)
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `block in protect'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `synchronize'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `protect'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:78:in `fetch'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:50:in `start_process'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:81:in `block (4 levels) in <top (required)>'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:33:in `block in for_each_process'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:27:in `times'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:27:in `for_each_process'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:80:in `block (3 levels) in <top (required)>'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:110:in `block in define_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/capistrano2.rb:96:in `block (3 levels) in <top (required)>'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/callback.rb:38:in `call'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `block in trigger'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `each'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `trigger'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:23:in `invoke_task_directly_with_callbacks'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:110:in `block in define_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/recipes/deploy.rb:453:in `block (2 levels) in load'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:46:in `block in execute_requested_actions'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:45:in `each'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:45:in `execute_requested_actions'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/help.rb:19:in `execute_requested_actions_with_help'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:34:in `execute!'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:14:in `execute'
    from /Users/ben/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/capistrano-2.15.5/bin/cap:4:in `<top (required)>'
    from ./.bundle/bin/cap:16:in `load'
    from ./.bundle/bin/cap:16:in `<main>'

$ ruby -v
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0]

$ rails -v
Rails 3.2.19

problem in version 0.3.5 of this gem. downgrading to 0.3.4 fixed it for me. happy to give more info.

😺

Running /usr/local/rbenv/bin/rbenv exec bundle exec sidekiqctl stop /data/apps/skindler/shared/tmp/pids/sidekiq.pid 10
--- Fails with stdout:nothing written.
It's that the PID isn't found, I'd bet it's some kind of race condition because it invokes sidekiq:quiet and checks (albeit with shell which I don't like but apparently works), and then it fails on this one telling me there's no PID.

Custom Monit templates

Hi,

Running bundle exec rails generate capistrano:sidekiq:monit:template resulted in

Could not find generator 'capistrano:sidekiq:monit:template'. Maybe you meant 'test_unit:controller' or 'test_unit:integration' or 'test_unit:generator'
Run `rails generate --help` for more options.

My intention is to tell Monit to restart Sidekiq if memory is exceeded. How do I use custom Monit templates? Thanks.

Run different queues on each role / server

I want to run a queue on my web server for some jobs and other jobs on a utility server. It doesn't look like this is possible right now, how would feel about me implementing that functionality in a similar way to specifying processes per role?

monit: action failed -- Other action already in progress

@seuros , I am constantly getting the following error. Any idea?

DEBUG[af8f40cf] Command: ( RBENV_ROOT=/.rbenv RBENV_VERSION=2.1.4 /usr/bin/env sudo /usr/bin/monit monitor sidekiq_workers_vimeo_production0 )
DEBUG[9161de6c] Command: ( RBENV_ROOT=/.rbenv RBENV_VERSION=2.1.4 /usr/bin/env sudo /usr/bin/monit monitor sidekiq_workers_vimeo_production0 )
DEBUG[25f6f1a5] monit: action failed -- Other action already in progress -- please try again later
DEBUG[af8f40cf] monit: action failed -- Other action already in progress -- please try again later
INFO[5b799faa] Finished in 0.098 seconds with exit status 0 (successful).
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host worker02.mll-worker.movielala.com: sudo exit status: 1

Missing capistrano monit tasks?

The readme mentions capistrano/sidekiq/monit tasks, but there seem to be missing from the git repo.

Typo

Seems that here should be a 'cmd' local variable

capistrano-sidekiq/lib/capistrano/tasks/capistrano2.rb

Line 61 in e03c69b

run cdm, roles: sidekiq_role, shell: "su - #{su_user}"

Example on front page not working?

I copy pasted the code on the front page in section

    Different number of processes per host

to my config/deploy/staging.rb and it seems to only span 1 processor for each host. Am I doing something wrong?

The code I have is

    set :stage, :staging
    set :pty,  false
    set :sidekiq_role, [:sidekiq_small, :sidekiq_big]
    set :sidekiq_big_processes, 8
    set :sidekiq_small_processes, 4
    server uk,  user: user, roles: [:sidekiq_big]
    server us,  user: user, roles: [:sidekiq_small]

And only one processor per host when we run cap staging sidekiq:restart

Multiple sidekiq apps in the same server

@seuros We love to divide our worker apps like data_services, notification, socialpost etc etc.

Does capistrano-sidekiq support multiple sidekiq process with multiple monit monitoring?

sidekiq_timeout overrides timeout set in config/sidekiq.yml

stop_process just takes sidekiq_timeout. It should look for a timeout setting in the sidekiq.yml, too.

tty Error

After updating gem. Now getting

DEBUG[3842d0cf]     sudo: no tty present and no askpass program specified
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host ***.***.365.192: sudo exit status: 1
sudo stdout: Nothing written
.
.
.

Tasks: TOP => sidekiq:monit:unmonitor
(See full trace by running task with --trace)
The deploy has failed with an error: #<SSHKit::Runner::ExecuteError: Exception while executing on host ***.***.365.192: sudo exit status: 1
sudo stdout: Nothing written
sudo stderr: sudo: no tty present and no askpass program specified

Not sure where the problem is

Not started sidekiq after deploy - if pty is true

** comment is not valid - REMOVED by author

first deploy : monit: action failed -- There is no service by that name

I am deploying a new sidekiq app ,

I got the following error.

INFO[f143e71d] Running /usr/bin/env sudo /usr/bin/monit unmonitor sidekiq_workers_vimeo_production0 on worker01.mll-worker.movielala.com
DEBUG[f143e71d] Command: ( RBENV_ROOT=~/.rbenv RBENV_VERSION=2.1.4 /usr/bin/env sudo /usr/bin/monit unmonitor sidekiq_workers_vimeo_production0 )
DEBUG[f143e71d] monit: action failed -- There is no service by that name
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host worker01.mll-worker.movielala.com: sudo exit status: 1

it is better put mothods in a library file to avoid scope conflicting

I used a capistrano plugin 'capistrano-sneakers' which is basicly forked from this 'capistrano-sidekiq'

And I encounter an error when I require both of them, the latter one will override the previous one's pid settings.

finally I found the root cause, due to this post:
http://stackoverflow.com/questions/6172292/scope-of-rakes-in-a-rails-project

so will you reorganize the code architecture to wrap the supporting methods
for_each_process
processes_pids
pid_process_exists?
pid_file_exists?

in a class?

sidekiq:quiet fails on initial deployment

I am using Capistrano 2.15.6 with capistrano-sidekiq 0.5.3.

sidekiq:quiet fails to run during initial deployment because bundle:install has not run yet so I receive the error message bundler: sidekiqctl not found. I tried using deploy:cold but it fails as well.

Bundler's integration with Capistrano runs bundle:install before deploy:finalize_update as seen here. I might switch to Capistrano 3 and capistrano-bundler at some point but not at this time.

Any suggestions?

is there an example how to launch multiple processes with different queues?

i want to launch 2 sidekiq daemons, one with
bundle exec sidekiq -q default -q low

and one with
bundle exec sidekiq -q high

is there any examples on how to achieve this with this gem?

Role filtering doesn't work correctly

I'm not 100% sure this gem is causing the problem I'm seeing, but a I can't see what else it would be.

I can run cap --role web production deploy successfully, but cap --role utility production deploy fails because it can't find the release on my web server. The only thing the utility role does is run capistrano-sidekiq: set :sidekiq_role, :utility.

This is my whole Capfile:

# Load DSL and Setup Up Stages
require 'capistrano/setup'

# Includes default deployment tasks
require 'capistrano/deploy'
require 'capistrano/puma'
require 'capistrano/bundler'
require 'capistrano/rails'
require 'capistrano/sidekiq'

# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('lib/capistrano/tasks/*.cap').each { |r| import r }

No such file or directory @ rb_sysopen

Sidekiq doesn't seem to be starting. This is the sidekiq log:

https://gist.github.com/sebastiaandegeus/67dc825fce4102bf13d3

ruby '2.2.1'
gem 'rails', '4.2.0'

Restarting Sidekiq on deploy:restart

I wanted to restart without deploying, so I ran a cap deploy:restart, assuming it would also restart Sidekiq; it does not seem to.

Is that just an oversight? It seems like if we automatically stop and restart on a deploy, then we should aslo be restarting on deploy:restart? At least by default? (From the code it looks like cap-sidekiq IS hooking into the deploy cap namespace, can't figure out why deploy:restart wouldn't be in the same 'namespace'.)

(Or maybe it's supposed to restart and there's something wrong with my setup.)

set :sidekiq_options will be overwritten by defaults if set too early

If you want to overwrite some default options, you need to delay that quite a bit, otherwise whatever you set will be overwritten by the gem's defaults again.

(not saying it's the best idea, but) I tried to overwrite :sidekiq_options in the Capfile itself as it totally works i.e. for other capistrano plugins like https://github.com/capistrano/rbenv, but no success. It took me way too long to figure out that the capistrano-sidekiq overwrites my custom settings because of the delayed execution of their defaults.

I'd suggest to change the behavior to the same https://github.com/capistrano/rbenv does, or at least a big warning in the readme.

Hope it saves other people tons of time!

Error on first time deploy

It seems that the sidekiq:quiet task is being called before the current symlink is created causing current_path to raise a directory not found error.

I tried disabling capistrano-sidekiq, doing the deploy, enabling capistrano-sidekiq again and deploying and it went fine.

May be for_each_process should use shared_path instead of current_path?

Sidekiq is not started after deployment

Sidekiq is not started after deployment is finished when using 0.5.3, but works fine on 0.5.2
It doesn't start it's workers after stopping in the beginning of the deployment.
And sometimes when it fails to stop them (probably), they keep running in the wrong directory until capistrano deleted the folders and all the errors are sent to Rollbar.

undefined method `namespace'

When I run cap <env> deploy, I'm getting the following error:

~/.rvm/gems/ruby-2.1.2/gems/capistrano-sidekiq-0.3.5/lib/capistrano/tasks/monit.cap:1:in `<top (required)>': undefined method `namespace' for main:Object (NoMethodError)

My stack:

Ruby 2.1.2
Rails 4.0.8
capistrano (2.15.5)
capistrano-sidekiq (0.3.5)

Multiple processes per host support in Monit

@seuros
Seems like one of the latests commit:
5745888

Reverted changes to support different processes count per each host. It was added in:
dcab28b

Was this intentional?

Multiple Queue per worker

Hi, first of all, thanks for publishing this gem.

How difficult would it be to implement a feature where we could have different workers per config file?

so that we can have for example, 2 workers:

worker 0:
pid file: shared_path/tmp/pids/sidekiq-0.pid (as usual)
config file: shared_path/config/sidekiq-0.yml
i.e. this worker would have queues: ":low" and ":normal" and 50 threads

worker 1:
pid file in shared_path/tmp/pids/sidekiq-1.pid
config file: shared_path/config/sidekiq-1.yml
this worker would have queues: ":priority" and ":urgent" and 25 threads. (specified in the sidekiq-1.yml)
...
and so on
...

that way we could have different workers serving different queues by priority and have different threads count.

Of course, this would be optional, keeping current functionality.
I would definitely like to help you on this feature.

What do you guys think?
cheers.

failed to run command 'bundle'

after the last update:

INFO [6f16c44a] Running /usr/bin/env nohup bundle exec sidekiq --index 0 --pidfile /home/myuser/apps/myapp/shared/tmp/pids/sidekiq.pid --environment staging --logfile /home/myuser/apps/myapp/shared/log/sidekiq.log --daemon on 54.72.xxx.xxx
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing on host 54.72.xxx.xxx: nohup exit status: 127
nohup stdout: Nothing written
nohup stderr: nohup: failed to run command ‘bundle’: No such file or directory

SSHKit::Command::Failed: nohup exit status: 127
nohup stdout: Nothing written
nohup stderr: nohup: failed to run command ‘bundle’: No such file or directory

Tasks: TOP => sidekiq:start
(See full trace by running task with --trace)
The deploy has failed with an error: Exception while executing on host 54.72.xxx.xxx: nohup exit status: 127
nohup stdout: Nothing written

sidekiq:quiet task causes failure if bundle install hasn't run yet

Using the sidekiq_default_hooks

  * 2014-06-05 13:36:15 executing `deploy:update_code'
    triggering before callbacks for `deploy:update_code'
  * 2014-06-05 13:36:15 executing `sidekiq:quiet'

Fails with:

/gems/bundler-1.3.5/lib/bundler/rubygems_integration.rb:214:in `block in replace_gem': sidekiq is not part of the bundle. Add it to Gemfile. (Gem::LoadError)
from /var/www/app/shared/bundle/ruby/1.9.1/bin/sidekiqctl:22:in `<main>'

Using:
capistrano (2.15.5)
capistrano-sidekiq (0.3.0)

Steps to replicate:

cap deploy a branch that doesn't include sidekiq in the bundle
switch to a branch that does include sidekiq, which also requires capistrano/sidekiq in the capistrano deploy file
attempt a cap deploy.

Workaround:

comment out the require for capistrano/sidekiq
deploy the branch that has sidekiq
uncomment the require
redeploy

Would this be fixable by changing the callback order so sidekiq:quiet executes after bundle?

  * 2014-06-05 13:41:36 executing `deploy:finalize_update'
    triggering before callbacks for `deploy:finalize_update'
  * 2014-06-05 13:41:36 executing `bundle:install'
    triggering before callbacks for `bundle:install'

If that's problematic, then maybe we can check if sidekiq is installed before sending the quiet signal?

`block in fetch': `stage' not found (IndexError)

I'm getting the following exception when I try to use capistrano-sidekiq. If I remove the require from Capfile, then my recipe works fine.

Any ideas of what might be causing this issue?

  * 2014-04-01 20:54:34 executing `sidekiq:start'
/Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:82:in `block in fetch': `stage' not found (IndexError)
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `block in protect'
        from <internal:prelude>:10:in `synchronize'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `protect'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:78:in `fetch'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:6:in `block (2 levels) in <top (required)>'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:87:in `call'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:87:in `block in fetch'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `block in protect'
        from <internal:prelude>:10:in `synchronize'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:110:in `protect'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/variables.rb:78:in `fetch'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:49:in `start_process'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:80:in `block (4 levels) in <top (required)>'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:33:in `block in for_each_process'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:27:in `times'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:27:in `for_each_process'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:79:in `block (3 levels) in <top (required)>'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:110:in `block in define_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bundler/gems/capistrano-sidekiq-0ddb0ead18e2/lib/capistrano/tasks/capistrano2.rb:95:in `block (3 levels) in <top (required)>'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/callback.rb:38:in `call'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `block in trigger'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `each'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:141:in `trigger'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:23:in `invoke_task_directly_with_callbacks'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:191:in `method_missing'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/namespaces.rb:110:in `block in define_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/recipes/deploy.rb:202:in `block (2 levels) in load'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:89:in `execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:46:in `block in execute_requested_actions'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:45:in `each'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:45:in `execute_requested_actions'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/help.rb:19:in `execute_requested_actions_with_help'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:34:in `execute!'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/lib/capistrano/cli/execute.rb:14:in `execute'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/gems/capistrano-2.15.5/bin/cap:4:in `<top (required)>'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bin/cap:23:in `load'
        from /Users/Fernando/.rvm/gems/ruby-1.9.3-p448-railsexpress/bin/cap:23:in `<main>'

Cannot start sidekiq with jruby-1.7.18

DEBUG[db8a581d] ObjectSpace is disabled; each_object will only work with Class, pass -X+O to enable
DEBUG[db8a581d]
DEBUG[db8a581d] org/jruby/RubyObjectSpace.java:173:in each_object' DEBUG[db8a581d] /data/web/api.plumlytics.com/shared/bundle/jruby/1.9/gems/sidekiq-3.3.0/lib/sidekiq/cli.rb:170:indaemonize'
DEBUG[db8a581d] /data/web/api.plumlytics.com/shared/bundle/jruby/1.9/gems/sidekiq-3.3.0/lib/sidekiq/cli.rb:41:in parse' DEBUG[db8a581d] /data/web/api.plumlytics.com/shared/bundle/jruby/1.9/gems/sidekiq-3.3.0/bin/sidekiq:7:in(root)'
DEBUG[db8a581d] org/jruby/RubyKernel.java:1087:in load' DEBUG[db8a581d] /data/web/api.plumlytics.com/shared/bundle/jruby/1.9/bin/sidekiq:23:in(root)'

gem is always using sudo

I have a setup where running monit commands does not require using sudo for deploy user. However capistrano-sidekiq uses sudo regardless.
capistrano/capistrano#920 says

If the gem is using sudo, then report it, as it shouldn't be doing, not at least without checking first.

question regarding deploy

do current busy workers get killed during the deploy?

I have long running workers but frequently need to deploy but do not want the workers killed.

And what I discovered is, although the worker is killed, it is not put in retries

Sidekiq does not restart

capistrano (2.15.5)
capistrano-sidekiq (0.3.4)
sidekiq (3.2.1)

@ster ➜  placeinfo rbenv:(2.1.2) git:(master) ✗ cap sidekiq:restart                                                                                                                   
  * 2014-07-21 18:16:30 executing `sidekiq:restart'
  * 2014-07-21 18:16:30 executing `sidekiq:stop'
  * executing "if [ -d /home/stereopark/projects/placeinfo/current ] && [ -f /home/stereopark/projects/placeinfo/shared/pids/sidekiq.pid ] && kill -0 `cat /home/stereopark/projects/placeinfo/shared/pids/sidekiq.pid`> /dev/null 2>&1; then cd /home/stereopark/projects/placeinfo/current && /home/stereopark/.rbenv/shims/bundle exec sidekiqctl stop /home/stereopark/projects/placeinfo/shared/pids/sidekiq.pid 10 ; else echo 'Sidekiq is not running'; fi"
    servers: ["146.185.162.57"]
    [146.185.162.57] executing command
 ** [out :: 146.185.162.57] Sidekiq is not running
    command finished in 297ms
  * 2014-07-21 18:16:31 executing `sidekiq:start'
  * executing "if [ -d /home/stereopark/projects/placeinfo/current ] && [ ! -f /home/stereopark/projects/placeinfo/shared/pids/sidekiq.pid ]; then cd /home/stereopark/projects/placeinfo/current ; /home/stereopark/.rbenv/shims/bundle exec sidekiq --index 0 --pidfile /home/stereopark/projects/placeinfo/shared/pids/sidekiq.pid --environment production --logfile /home/stereopark/projects/placeinfo/shared/log/sidekiq.log --daemon ; else echo 'Sidekiq is already running'; fi"
    servers: ["146.185.162.57"]
    [146.185.162.57] executing command
 ** [out :: 146.185.162.57] Sidekiq is already running
    command finished in 124ms

pidfile handling

Sidekiq currently uses two different methods to generate the name for pidfiles: If only one Sidekiq process is running the file is only called sidekiq.pid, if two or more processes are running the pid files are called sidekiq-0.pid, sidekiq-1.pid, ... This leads to problems in deployment, as changing from 1 to 2 processes or back to 1 always leads to not stopped sidekiq workers.

So when e.g. changing the sidekiq_processes from 1 to 2 not the process stored in the sidekiq.pid is stopped but the non-existing processes sidekiq-0/1.pid. So the old process keeps running.

Sidekiq does not start

$ cap production sidekiq:start --trace
** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
Identity added: /Users/Villins/.ssh/id_rsa (/Users/Villins/.ssh/id_rsa)
** Invoke rvm:hook (first_time)
** Invoke passenger:rvm:hook (first_time)
** Invoke passenger:test_which_passenger (first_time)
** Execute passenger:test_which_passenger
** Execute passenger:rvm:hook
** Execute rvm:hook
DEBUG [c3f02cfb] Running /usr/bin/env [ -d ~/.rvm ] as [email protected]
DEBUG [c3f02cfb] Command: [ -d ~/.rvm ]
DEBUG [c3f02cfb] Finished in 0.894 seconds with exit status 0 (successful).
** Invoke rvm:check (first_time)
** Execute rvm:check
DEBUG [4aec0ce3] Running ~/.rvm/bin/rvm version as [email protected]
DEBUG [4aec0ce3] Command: ~/.rvm/bin/rvm version
DEBUG [4aec0ce3]    rvm 1.25.28 (master) by Wayne E. Seguin <[email protected]>, Michal Papis <[email protected]> [https://rvm.io/]
DEBUG [4aec0ce3] Finished in 0.280 seconds with exit status 0 (successful).
rvm 1.25.28 (master) by Wayne E. Seguin <[email protected]>, Michal Papis <[email protected]> [https://rvm.io/]
DEBUG [3a07d67d] Running ~/.rvm/bin/rvm current as [email protected]
DEBUG [3a07d67d] Command: ~/.rvm/bin/rvm current
DEBUG [3a07d67d]    ruby-2.1.2@danche
DEBUG [3a07d67d] Finished in 0.273 seconds with exit status 0 (successful).
ruby-2.1.2@danche
DEBUG [677ee191] Running ~/.rvm/bin/rvm default do ruby --version as [email protected]
DEBUG [677ee191] Command: ~/.rvm/bin/rvm default do ruby --version
DEBUG [677ee191]    ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
DEBUG [677ee191] Finished in 0.403 seconds with exit status 0 (successful).
ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
** Invoke bundler:map_bins (first_time)
** Invoke passenger:bundler:hook (first_time)
** Execute passenger:bundler:hook
** Execute bundler:map_bins
** Invoke deploy:set_rails_env (first_time)
** Execute deploy:set_rails_env
** Invoke deploy:set_rails_env
** Invoke sidekiq:start (first_time)
** Execute sidekiq:start

bundle exec cap production sidekiq:monit:config copies to / rather than /etc/monit/conf.d

I run bundle exec cap production sidekiq:monit:config
Config file is copied to /sidekiq_xxx_production.conf
But my monit recipes are located at; /etc/monit/conf.d/sidekiq_xxx_production.conf
Is there a configuration parameter that I am missing so set the path to monit recipes? I haven't been able to find any documentation about this.

I am using this in combination with the Talking Quickly rails server template, capistrano recipes, and book.

Inspeqtor support?

I see you have monit support now. Anyone want to add Inspeqtor support? Ideally you'd integrate #51 init support too because Inspeqtor does not start/stop processes on its own.

To integrate Inspeqtor, you just need to signal the start and stop of the deploy so Inspeqtor will ignore any process changes by running the following commands:

inspeqtorctl start deploy
inspeqtorctl finish deploy

Sidekiq process with scheduled jobs does not die?

I'm running sidekiq on several setups, but seeing that sidekiq processes consistently do not die on a machine that has lots of "scheduled jobs" (ie jobs scheduled to be performed at a future date via perform_in).

Seen in the sidekiq.log:

2014-04-15T21:47:16Z [pid] ... DEBUG: Terminating 4 actors...
...
2014-04-15T21:47:26Z [pid] ... ERROR: Couldn't cleanly terminate all actors in 10 seconds!

This means that after most deploys we end up killing old sidekiq processes by hand, otherwise we get tons of sidekiq processes and unicorns get slow. (But the processes do respond nicely to a kill command.)

Is this an issue with sidekiq-ctrl not responding to the kill -0 command while it has scheduled jobs or something in the capinstrano integration?

Not started sidekiq after deploy

Sidekiq stopped successfully, but not started after deploy

sidekiq upstart:

description "Sidekiq Background Workers"
start on runlevel [2345]
stop on runlevel [06]
normal exit 0 TERM USR1
respawn
respawn limit 3 30
exec su - deployer -c 'cd /home/deployer/name/current && exec bundle exec sidekiq -e production -P /home/deployer/name/shared/tmp/pids/sidekiq.pid'

added to Capfile

require 'capistrano/sidekiq'

Cannot start the sidekiq by capistrano3

I know there are many people have solved this problem .

But it does't work for me

Here is my Capfile

# Load DSL and set up stages
require 'capistrano/setup'

# Include default deployment tasks
require 'capistrano/deploy'
require 'capistrano/rvm'
require 'capistrano/bundler'
require 'capistrano/rails/assets'
require 'capistrano/rails/migrations'
require 'capistrano/sidekiq'
require 'capistrano/sidekiq/monit'


# Load custom tasks from `lib/capistrano/tasks' if you have any defined
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }

deploy.rb

# config valid only for current version of Capistrano
lock '3.3.5'

set :application, 'xshare'
set :repo_url, '[email protected]:linjunzhugg/xshare.git'

# Default branch is :master
# ask :branch, proc { `git rev-parse --abbrev-ref HEAD`.chomp }.call

# Default deploy_to directory is /var/www/my_app_name
set :deploy_to, '/home/deployer/xshare'

# Default value for :scm is :git
set :scm, :git

set :linked_files, %w{config/database.yml config/share_oauth2.yml config/secrets.yml}

set :linked_dirs, %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system}

set :rvm_ruby_version, '[email protected]'

set :pty,  false

namespace :deploy do

  after :restart, :clear_cache do
    on roles(:web), in: :groups, limit: 3, wait: 10 do
      # Here we can do anything such as:
      # within release_path do
      #   execute :rake, 'cache:clear'
      # end
    end
  end

end

Enviroment:

Rails 4.1.5
Ruby 2.0
Capistrano 3

Sidekiq does not restart on deploy

Sidekiq does not reliably respawn on each deploy when I define concurrency options for sidekiq.

My environment:
capistrano-sidekiq (0.1.3)
sidekiq (3.0.0)
capistrano (3.0.1)
rails ( 4.0.3)

I require the gem in my Capfile using require 'capistrano/sidekiq'.

My production deploy file has:

namespace :deploy do
  after :starting,  "sidekiq:stop"
  after :finishing, "sidekiq:cleanup"
  after :finishing, "sidekiq:respawn"
end

I have the following in config/sidekiq.yml:


---
:concurrency: 5

I've gisted the relevant output from the first deploy, as well as an example of subsequent deploys here. After the first deploy I can see the PID for sidekiq:

deploy     334  4.5 11.4 849724 117060 ?       Sl   19:51   0:02 sidekiq 3.0.0 myapp[0 of 5 busy]

Nothing appears after a subsequent deploy(s).

Development group

I take it this is only needed in development so in gemfile:

gem "capistrano-sidekiq", group: :development

0.5.1 Changes?

Just saw that 0.5.1 was released yesterday, but without any changelog updates, could you take a moment to add those?

Its not loading default config/sidekiq.yml file

I have specified my settings in config/sidekiq.yml file.


---
:concurrency: 1
production:
  :concurrency: 25
:queues:
  - [track_event, 5]
  - [track_charge, 1]
  - [track_user, 1]
  - [admin_email, 1]

Its not loading it and because of that no process is working.. And there are zero workers in my sidekiq dashboard.

Solution to setting environment variables

I'm looking for a solution to set some environmental variables prior to sidekiq starting.

I've tried using dotenv but it didn't pick up my .env or .env.production file.
I've tried adding my ENV (source of all my variables) to my $HOME/.profile and it still didn't pick up those.

I've been testing by reading /proc/$PID/environ

Ideally I don't want to keep my sensitive data in deploy.rb file or inside my repo.

Any ideas?

Sidekiq fails to start via deploy, but does start via sidekiq:start

When manually calling the start task, i.e. cap staging sidekiq:start, everything works just like it's supposed to. Sidekiq starts and everything is wonderful. However, the deploy task fails to start Sidekiq.

From cap staging sidekiq:start:

INFO [bd06ac04] Running /opt/rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/apps/my_app/current/tmp/sidekiq-0.pid --environment staging --logfile /home/apps/my_app/shared/log/sidekiq.log --daemon as deploy@myhost

From sidekiq.log:

2015-04-14T15:04:28.970Z 15803 TID-ox6iueuu0 INFO: Booting Sidekiq 3.3.3 with redis options {:url=>"redis://xx.xx.xx.xx:6379/0"}
2015-04-14T15:04:29.774Z 15803 TID-ox6iueuu0 INFO: Running in ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]
2015-04-14T15:04:29.774Z 15803 TID-ox6iueuu0 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2015-04-14T15:04:29.774Z 15803 TID-ox6iueuu0 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org/pro
2015-04-14T15:04:30.112Z 15895 TID-ouecgm9i0 INFO: Booting Sidekiq 3.3.3 with redis options {:url=>"redis://xx.xx.xx.xx:6379/0"}
2015-04-14T15:04:30.924Z 15895 TID-ouecgm9i0 INFO: Running in ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]
2015-04-14T15:04:30.924Z 15895 TID-ouecgm9i0 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2015-04-14T15:04:30.924Z 15895 TID-ouecgm9i0 INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org/pro

From cap staging deploy:

INFO [fec5414e] Running /opt/rbenv/bin/rbenv exec bundle exec sidekiq --index 0 --pidfile /home/apps/my_app/current/tmp/sidekiq-0.pid --environment staging --logfile /home/apps/my_app/shared/log/sidekiq.log --daemon as deploy@myhost

From sidekiq.log:

No such file or directory @ rb_sysopen - /home/apps/my_app/current/tmp/sidekiq-0.pid
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `initialize'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `open'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `write_pid'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:42:in `parse'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/bin/sidekiq:7:in `<top (required)>'
/home/apps/my_app/shared/bundle/ruby/2.2.0/bin/sidekiq:23:in `load'
/home/apps/my_app/shared/bundle/ruby/2.2.0/bin/sidekiq:23:in `<main>'
No such file or directory @ rb_sysopen - /home/apps/my_app/current/tmp/sidekiq-1.pid
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `initialize'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `open'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:352:in `write_pid'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/lib/sidekiq/cli.rb:42:in `parse'
/home/apps/my_app/shared/bundle/ruby/2.2.0/gems/sidekiq-3.3.3/bin/sidekiq:7:in `<top (required)>'
/home/apps/my_app/shared/bundle/ruby/2.2.0/bin/sidekiq:23:in `load'
/home/apps/my_app/shared/bundle/ruby/2.2.0/bin/sidekiq:23:in `<main>'

It seems like the same command is being run in both cases, but clearly something different is going on. Am I just misunderstanding something? Please let me know if you need any more information to help clarify the situation.

Note: The error message I'm receiving matches #71, but I'm opening a new issue as the behavior I'm seeing is a little different, since Sidekiq will start if called by itself.

How do I configure url server and client ?

Hi I would like to configure this values below in config/deploy/production.rb

  Sidekiq.configure_server do |config|
    config.redis = {url: 'redis://localhost:6379/0', namespace: 'app'}
  end

  Sidekiq.configure_client do |config|
    config.redis = {url: 'redis://localhost:6379/0', namespace: 'app'}
  end

Some thing like

set :sidekiq_log, "#{shared_path}/log/sidekiq.log"
#set HOW CONFIG URL AND NAMESPACE??