Comments (27)
snmgian
commented on May 25, 2024
1
Glad to there that! I'll merge the PR so.
As a side note, -C option is for passing a .yml file. If something like config/sidekiq.rb is passed, sidekiq will fail to start. You didn't experience that problem because the file didn't exist in your setup :)
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Is it because reports worker payment are each an instance of sidekiq running and they each use sidekiq.rb which in turn loads the schedule into each instance?
reports: bundle exec sidekiq -q reports
worker: bundle exec sidekiq -q worker
payment: bundle exec sidekiq -q payment
from sidekiq-scheduler.
elpic
commented on May 25, 2024
Hi @maxrosecollins,
Are you sure the jobs is being schedule on each queue? Maybe the message "INFO: queueing Testing (testing)" is the thing that is wrong.
Because I run a similar example and the jobs only run on his own queue.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Hi @elpic,
It definitely queues it on each queue.
For some reason if I do the following it only queues it on the worker queue. The file ./config/sidekiq.rb doesn't exist by the way...
reports: bundle exec sidekiq -q reports -C ./config/sidekiq.rb
worker: bundle exec sidekiq -q worker
payment: bundle exec sidekiq -q payment -C ./config/sidekiq.rb
from sidekiq-scheduler.
elpic
commented on May 25, 2024
Which version of sidekiq-scheduler are you using?
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
I'm using 2.0.4
from sidekiq-scheduler.
elpic
commented on May 25, 2024
Could you try v2.0.5?
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
I tried with 2.0.5.
When I start up the server I'm getting the following. Which runs the job three times. Once on each instance of sidekiq
11:43:57 mailer.1 | 2016-03-10T11:43:57.660Z 70723 TID-owkmjau4o INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
11:43:57 mailer.1 | 2016-03-10T11:43:57.660Z 70723 TID-owkmjau4o INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
11:43:57 mailer.1 | 2016-03-10T11:43:57.665Z 70723 TID-owkmjau4o INFO: Loading Schedule
11:43:57 mailer.1 | 2016-03-10T11:43:57.666Z 70723 TID-owkmjau4o INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
11:43:57 worker.1 | 2016-03-10T11:43:57.669Z 70722 TID-oxod2nf24 INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
11:43:57 worker.1 | 2016-03-10T11:43:57.669Z 70722 TID-oxod2nf24 INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
11:43:57 reports.1 | 2016-03-10T11:43:57.669Z 70721 TID-ougyo0w00 INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
11:43:57 reports.1 | 2016-03-10T11:43:57.670Z 70721 TID-ougyo0w00 INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
11:43:57 worker.1 | 2016-03-10T11:43:57.673Z 70722 TID-oxod2nf24 INFO: Loading Schedule
11:43:57 worker.1 | 2016-03-10T11:43:57.673Z 70722 TID-oxod2nf24 INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
11:43:57 reports.1 | 2016-03-10T11:43:57.674Z 70721 TID-ougyo0w00 INFO: Loading Schedule
11:43:57 reports.1 | 2016-03-10T11:43:57.674Z 70721 TID-ougyo0w00 INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
11:43:57 mailer.1 | 2016-03-10T11:43:57.687Z 70723 TID-owkmjau4o INFO: Schedules Loaded
11:43:57 mailer.1 | 2016-03-10T11:43:57.688Z 70723 TID-owkmjau4o INFO: Starting processing, hit Ctrl-C to stop
11:43:57 mailer.1 | 2016-03-10T11:43:57.688Z 70723 TID-owkmphnmk INFO: ReliableFetch activated
11:43:57 worker.1 | 2016-03-10T11:43:57.691Z 70722 TID-oxod2nf24 INFO: Schedules Loaded
11:43:57 reports.1 | 2016-03-10T11:43:57.692Z 70721 TID-ougyo0w00 INFO: Schedules Loaded
11:43:57 worker.1 | 2016-03-10T11:43:57.692Z 70722 TID-oxod2nf24 INFO: Starting processing, hit Ctrl-C to stop
11:43:57 worker.1 | 2016-03-10T11:43:57.693Z 70722 TID-oxodzrez0 INFO: ReliableFetch activated
11:43:57 reports.1 | 2016-03-10T11:43:57.693Z 70721 TID-ougyo0w00 INFO: Starting processing, hit Ctrl-C to stop
11:43:57 reports.1 | 2016-03-10T11:43:57.693Z 70721 TID-ougyq7x28 INFO: ReliableFetch activated
from sidekiq-scheduler.
elpic
commented on May 25, 2024
Can you tests the following:
Change the cron to one minute and add a puts on the worker
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
yep I will try that now!
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Hi It still runs three times. the log output
12:59:00 mailer.1 | 2016-03-10T12:59:00.136Z 74592 TID-ovqv3q7ys INFO: queueing DailyCheckWorker (DailyCheckWorker)
12:59:00 reports.1 | 2016-03-10T12:59:00.136Z 74589 TID-ouwzgmxqw INFO: queueing DailyCheckWorker (DailyCheckWorker)
12:59:00 worker.1 | 2016-03-10T12:59:00.136Z 74590 TID-ovp0krgt8 INFO: queueing DailyCheckWorker (DailyCheckWorker)
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmb1k DailyCheckWorker JID-ced35f10a346de83043868b2 INFO: start
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmats DailyCheckWorker JID-3694191cd36ac190ae730880 INFO: start
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmb1k DailyCheckWorker JID-ced35f10a346de83043868b2 INFO: IM RUNNING!!!
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmamk DailyCheckWorker JID-f6f80bb6dd945ecaedd6efd6 INFO: start
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmats DailyCheckWorker JID-3694191cd36ac190ae730880 INFO: IM RUNNING!!!
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmb1k DailyCheckWorker JID-ced35f10a346de83043868b2 INFO: done: 0.001 sec
12:59:00 worker.1 | 2016-03-10T12:59:00.216Z 74590 TID-ovp1fmamk DailyCheckWorker JID-f6f80bb6dd945ecaedd6efd6 INFO: IM RUNNING!!!
12:59:00 worker.1 | 2016-03-10T12:59:00.216Z 74590 TID-ovp1fmats DailyCheckKeywordWorker JID-3694191cd36ac190ae730880 INFO: done: 0.001 sec
12:59:00 worker.1 | 2016-03-10T12:59:00.216Z 74590 TID-ovp1fmamk DailyCheckKeywordWorker JID-f6f80bb6dd945ecaedd6efd6 INFO: done: 0.001 sec
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
When you tried did you start three instances of sidekiq like so?
reports: bundle exec sidekiq -q reports
worker: bundle exec sidekiq -q worker
payment: bundle exec sidekiq -q payment
from sidekiq-scheduler.
elpic
commented on May 25, 2024
Yes, but not using foreman, I will try with foreman
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Okay.
If I start sidekiq like this
reports: bundle exec sidekiq -q reports -C ./config/sidekiq.rb
worker: bundle exec sidekiq -q worker
payment: bundle exec sidekiq -q payment -C ./config/sidekiq.rb
Then I see the following when I start the app. It says it is only loaded once and it runs only one
13:11:16 reports.1 | 2016-03-10T13:11:16.642Z 76006 TID-oxhur4wsk INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
13:11:16 worker.1 | 2016-03-10T13:11:16.642Z 76007 TID-owlhm1czg INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
13:11:16 reports.1 | 2016-03-10T13:11:16.642Z 76006 TID-oxhur4wsk INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
13:11:16 worker.1 | 2016-03-10T13:11:16.642Z 76007 TID-owlhm1czg INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
13:11:16 mailer.1 | 2016-03-10T13:11:16.642Z 76008 TID-ougpne328 INFO: Running in ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-darwin14]
13:11:16 mailer.1 | 2016-03-10T13:11:16.642Z 76008 TID-ougpne328 INFO: Sidekiq Pro 3.0.5, commercially licensed. Thanks for your support!
13:11:16 reports.1 | 2016-03-10T13:11:16.644Z 76006 TID-oxhur4wsk INFO: Loading Schedule
13:11:16 reports.1 | 2016-03-10T13:11:16.644Z 76006 TID-oxhur4wsk INFO: Schedule empty! Set Sidekiq.schedule
13:11:16 mailer.1 | 2016-03-10T13:11:16.644Z 76008 TID-ougpne328 INFO: Loading Schedule
13:11:16 mailer.1 | 2016-03-10T13:11:16.644Z 76008 TID-ougpne328 INFO: Schedule empty! Set Sidekiq.schedule
13:11:16 reports.1 | 2016-03-10T13:11:16.644Z 76006 TID-oxhur4wsk INFO: Schedules Loaded
13:11:16 mailer.1 | 2016-03-10T13:11:16.644Z 76008 TID-ougpne328 INFO: Schedules Loaded
13:11:16 mailer.1 | 2016-03-10T13:11:16.646Z 76008 TID-ougpne328 INFO: Starting processing, hit Ctrl-C to stop
13:11:16 reports.1 | 2016-03-10T13:11:16.646Z 76006 TID-oxhur4wsk INFO: Starting processing, hit Ctrl-C to stop
13:11:16 reports.1 | 2016-03-10T13:11:16.646Z 76006 TID-oxhvauua0 INFO: ReliableFetch activated
13:11:16 mailer.1 | 2016-03-10T13:11:16.646Z 76008 TID-ougqopf6o INFO: ReliableFetch activated
13:11:16 worker.1 | 2016-03-10T13:11:16.650Z 76007 TID-owlhm1czg INFO: Loading Schedule
13:11:16 worker.1 | 2016-03-10T13:11:16.650Z 76007 TID-owlhm1czg INFO: Scheduling DailyCheckWorker {"cron"=>"14 * * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
13:11:16 worker.1 | 2016-03-10T13:11:16.919Z 76007 TID-owlhm1czg INFO: Schedules Loaded
13:11:16 worker.1 | 2016-03-10T13:11:16.921Z 76007 TID-owlhm1czg INFO: Starting processing, hit Ctrl-C to stop
13:11:16 worker.1 | 2016-03-10T13:11:16.921Z 76007 TID-owli0p6bs INFO: ReliableFetch activated
and
13:14:00 worker.1 | 2016-03-10T13:14:00.183Z 76648 TID-ouog8yz90 INFO: queueing DailyCheckWorker (DailyCheckWorker)
13:14:00 worker.1 | 2016-03-10T13:14:00.186Z 76648 TID-ouoh4qa64 DailyCheckWorker JID-227e79851e19e210d25e1eca INFO: start
13:14:00 worker.1 | 2016-03-10T13:14:00.186Z 76648 TID-ouoh4qa64 DailyCheckWorker JID-227e79851e19e210d25e1eca INFO: IM RUNNING!!!
13:14:00 worker.1 | 2016-03-10T13:14:00.186Z 76648 TID-ouoh4qa64 DailyCheckWorker JID-227e79851e19e210d25e1eca INFO: done: 0.0 sec
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@elpic Did you manage to recreate the issue?
from sidekiq-scheduler.
snmgian
commented on May 25, 2024
Hi @maxrosecollins.
sidekiq-scheduler does not check which queues are enabled, so each of your sidekiq instances schedule DailyCheckWorker:
11:43:57 mailer.1 | 2016-03-10T11:43:57.666Z 70723 TID-owkmjau4o INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
11:43:57 worker.1 | 2016-03-10T11:43:57.673Z 70722 TID-oxod2nf24 INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
11:43:57 reports.1 | 2016-03-10T11:43:57.674Z 70721 TID-ougyo0w00 INFO: Scheduling DailyCheckWorker {"cron"=>"3 10 * * *", "class"=>"DailyCheckWorker", "queue"=>"worker", "description"=>"This schedules the checks"}
and then the worker instance executes those scheduled jobs:
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmb1k DailyCheckWorker JID-ced35f10a346de83043868b2 INFO: IM RUNNING!!!
12:59:00 worker.1 | 2016-03-10T12:59:00.215Z 74590 TID-ovp1fmats DailyCheckWorker JID-3694191cd36ac190ae730880 INFO: IM RUNNING!!!
12:59:00 worker.1 | 2016-03-10T12:59:00.216Z 74590 TID-ovp1fmamk DailyCheckWorker JID-f6f80bb6dd945ecaedd6efd6 INFO: IM RUNNING!!!
I'll work on fixing this.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Hi @snmgian.
I thought that might be the case because when I pass a empty config to the instances I don't want to run the worker only the instance with a config runs. If that makes sense.
from sidekiq-scheduler.
snmgian
commented on May 25, 2024
@maxrosecollins, I just pushed a PR: #96
Could you point your Gemfile's entry to PR's commit and verify that the issue is finally solved?
Although I reproduced the issue and found the way to solve it, I would prefer to verify if it's also solved under your setup. Thanks.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@snmgian sure thing I will give it a try now
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@snmgian How do I point it at a PR?
I tried gem 'sidekiq-scheduler', branch: 'issues/91' and pull/96 in my Gemfile
from sidekiq-scheduler.
snmgian
commented on May 25, 2024
gem 'sidekiq-scheduler', git: 'https://github.com/snmgian/sidekiq-scheduler.git', branch: 'issues/91'
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@snmgian Thanks.
It works for me too! Thank you for adding this functionality. Much appreciated.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@snmgian Ah thanks for the tip.
Do i need to wait for a release?
from sidekiq-scheduler.
snmgian
commented on May 25, 2024
Although we don't have a release date yet, we are planning to release soon.
The code change (PR #96) is a kind of breaking one, so probably a config flag will be added to enable this behavior. I'll keep you in touch if you need to make some config changes.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
@snmgian okay no problem. Thanks again
from sidekiq-scheduler.
snmgian
commented on May 25, 2024
A new version has been released, it's available through rubygems. As PR #96 is a breaking change, a configuration flag has been added.
In your sidekiq.yml, you'll need to add this:
:scheduler:
:listened_queues_only: trueThe default value of that flag is false.
from sidekiq-scheduler.
maxrosecollins
commented on May 25, 2024
Hi @snmgian
Great news, thank you!
from sidekiq-scheduler.
Related Issues (20)
- Did sidekiq-scheduler 5.0.0 add a Rails dependency? HOT 3
- Raises an exception when rufus_scheduler_options is nil by default HOT 3
- Could not find gem 'sidekiq-scheduler (= 5.0.1)' with platform 'arm64-darwin-21' HOT 1
- Sidekiq version requirement HOT 1
- Run job multiple times, but never at the same time HOT 3
- Uppercase character in header name: Cache-Control
- Sidekiq 7.1 warns when deprecated Redis commands are used
- Cron expression with day of month does not work HOT 4
- Support for Ruby 3.0 and deconstruct Hashes as keyword arguments. HOT 5
- Move away from deprecated Redis commands HOT 2
- undefined method `next_time' for nil:NilClass HOT 5
- re-use Sidekiq Rack::Static instance for assets
- RedisManager does not appear to be updating in test env HOT 6
- Sidekick (docker) deprecation warning HOT 1
- UI not loading on heroku HOT 7
- Error handler threw an error HOT 2
- [feature request] environment specific schedules HOT 1
- Recommend Sidekiq::Job instead of Sidekiq::Worker HOT 1
- How to test that sidekiq.yml is *correct*? HOT 3
- SidekiqScheduler::Manager enormous inspect output HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sidekiq-scheduler.