Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exit after a fixed number of jobs executed. #1164

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

bretweinraub
Copy link

@bretweinraub bretweinraub commented Mar 4, 2022

Add command line switch (--exit-after) or Environment
variable (EXIT_AFTER),which causes work to exit after a specific
number of jobs, irrespective of success or failure.

varaible (EXIT_AFTER),which causes work to exit after a specific
number of jobs, irrespective of success or failure.
@albus522
Copy link
Member

What is the problem you are trying to solve with this?

@bretweinraub
Copy link
Author

This solves a lot of devops issues that arise from using god or monit, for example:

https://github.com/collectiveidea/delayed_job/blob/master/contrib/delayed_job.monitrc

A whole bunch of problems go away:

  • I don't need to run god or monit, I just use cron and a launch script that checks for the previously running iteration of delayed job
  • I don't need to worry about memory leaks, since the process recycles and restarts every so often.
  • I don't need to worry about what delayed job should do when a new version of the app is deployed, since the process recycles and restarts every so often.

So our devops labor went from DJ being the biggest pain point in our infra to essentially zero by switching to the cron/launch-script model. Getting rid of monit has been amazing.

@albus522
Copy link
Member

There are many gotchas and issues to what you are doing that I would not generally advise it. But if it is something you want to continue doing, there is a mechanism you can use that can do what you want without monkey patching from directly within your app. Delayed Job has a plugin infrastructure that we need to better document.

Putting this in an initializer should accomplish what you want:

class QuitAfterXJobs < Delayed::Plugin
  callbacks do |lifecycle|
    if ENV["EXIT_AFTER"]
      lifecycle.around(:perform) do |worker, job, &block|
        result = block.call

        @total_jobs_run ||= 0
        @total_jobs_run += 1
        worker.stop if @total_jobs_run > ENV["EXIT_AFTER"].to_i

        result
      end
    end
  end
end

Delayed::Worker.plugins << [QuitAfterXJobs]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants