Grace time is not honoured

sidekiq-worker-killer

Sidekiq is probably the best background processing framework today. At the same time, memory leaks are very hard to tackle in Ruby and we often find ourselves with growing memory consumption. Instead of spending herculean effort fixing leaks, why not kill your processes when they got to be too large?

Highly inspired by Gitlab Sidekiq MemoryKiller and Noxa Sidekiq killer.

NOTE

This gem needs to get the used memory of the Sidekiq process. For this we use GetProcessGem, but be aware that if you are running Sidekiq in Heroku(or any container) the memory usage will not be accurate.

quick-refs: install | usage | available options | development

Install

Use Bundler

gem "sidekiq-worker-killer"

Usage

Add this to your Sidekiq configuration.

require 'sidekiq/worker_killer'

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Sidekiq::WorkerKiller, max_rss: 480
  end
end

Available options

The following options can be overridden.

Option	Defaults	Description
max_rss	0 MB (disabled)	Max RSS in megabytes used by the Sidekiq process. Above this, shutdown will be triggered.
grace_time	900 seconds	When shutdown is triggered, the Sidekiq process will not accept new job and wait at most 15 minutes for running jobs to finish. If Float::INFINITY specified, will wait forever.
shutdown_wait	30 seconds	When the grace time expires, still running jobs get 30 seconds to stop. After that, kill signal is triggered.
kill_signal	SIGKILL	Signal to use to kill Sidekiq process if it doesn't stop.
gc	true	Try to run garbage collection before Sidekiq process stops in case of exceeded max_rss.
skip_shutdown_if	proc {false}	Executes a block of code after max_rss exceeds but before requesting shutdown.
on_shutdown	nil	Executes a block of code before just before requesting shutdown. This can be used to send custom logs or metrics to external services.

skip_shutdown_if is expected to return anything other than false or nil to skip shutdown.

require 'sidekiq/worker_killer'

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add Sidekiq::WorkerKiller, max_rss: 480, skip_shutdown_if: ->(worker, job, queue) do
      worker.to_s == 'LongWorker'
    end
  end
end

Development

Pull Requests are very welcome!

There are tasks that may help you along the way in a makefile:

make console # Loads the whole stack in an IRB session.
make test # Run tests.
make lint # Run rubocop linter.

Please make sure that you have tested your code carefully before opening a PR, and make sure as well that you have no style issues.

Authors

See the list of contributors who participated in this project.

License

Please see LICENSE

	def wait_job_finish_in_grace_time
	start = Time.now
	loop do
	break if grace_time_exceeded?(start)
	break if no_jobs_on_quiet_processes?
	sleep(1)
	end
	end

	def no_jobs_on_quiet_processes?
	Sidekiq::ProcessSet.new.each do \|process\|
	return false if !process["busy"].zero? && process["quiet"]
	end
	true
	end

	def no_jobs_on_quiet_processes?
	Sidekiq::ProcessSet.new.each do \|process\|
	return false if !process["busy"] == 0 && process["quiet"]
	end
	true
	end

klaxit / sidekiq-worker-killer Goto Github PK

sidekiq-worker-killer's Introduction

sidekiq-worker-killer

Install

Usage

Available options

Development

Authors

License

sidekiq-worker-killer's People

Contributors

Stargazers

Watchers

Forkers

sidekiq-worker-killer's Issues

Recommend Projects

Recommend Topics

Recommend Org