ActiveJobResqueSolo
A plugin for ActiveJob with Resque to prevent duplicate enqueuing of jobs.
Installation
Add this line to your application's Gemfile:
gem 'active_job_resque_solo'
And then execute:
$ bundle
Or install it yourself as:
$ gem install active_job_resque_solo
Usage
In your job class, include the plugin:
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
queue_as :default
def perform(*args); end
end
If an instance of the job with matching arguments is either waiting for a worker or currently executing, Solo will prevent a new instance of the job from being enqueued.
You can control which named arguments are used to determine uniqueness in the queue:
solo_only_args
solo_except_args
solo_any_args
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
# Only compare "user" args when checking for duplicate jobs.
solo_only_args :user
queue_as :default
def perform(user:, nonce:)
end
end
Conversely, you can exclude arguments from being checked for duplicates. This is useful when plugins add arguments of their own to your jobs.
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
solo_except_args :nonce, :retry_count
queue_as :default
def perform(user:, nonce:)
end
end
Specify solo_any_args
to allow only one instance of your job to be enqueued or executing at any given
time regardless of the arguments used in each instance.
solo_any_args
overrides solo_only_args
and solo_except_args
.
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
solo_any_args
queue_as :default
def perform(user: nil)
end
end
Locking
Solo uses an internal locking mechanism to prevent multiple processes from enqueuing the same job during race conditions. This gem does not perform any locking around job execution.
The lock prevents competing jobs of the same class and arguments from being
enqueued, complying with the argument filtering programmed with solo_only_args
and solo_except_args
.
The default Redis key prefix is "ajr_solo". It can be set to a different,
arbitrary string of your choice using solo_lock_key_prefix
:
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
solo_lock_key_prefix "my_lock_prefix"
def perform(*args)
end
end
Re-enqueueing from within the job
The default behavior of this gem allows a Job to re-enqueue itself while it is
executing. If you know that your job should not be re-enqueueing itself, you can
prevent inadvertent re-enqueueing of the job by using solo_self_enqueueing :prevent
.
Jobs that fail
or raise
an error may be retried by the framework, and are not
affected by this option.
class MyJob < ActiveJob::Base
include ActiveJob::Plugins::Resque::Solo
solo_self_enqueueing :prevent
def perform(*args)
MyJob.perform_later # <== this will not enqueue a job because of :prevent, above.
raise MyError # <== retries are still allowed
end
end
Duplicate enqueues are possible
While this plugin will greatly reduce duplicate instances of a job from being enqueued, a job may be enqueued multiple times if the Redis response times are very slow. Slowness could be caused by extremely high load on Redis or networking issues.
Since duplicate enqueueing of jobs is a possibility, make sure that that your code does not rely on this gem for sensitive, critical sections of code that must not be processed more than once.
The locks are acquired for dynamic amounts of time, but expire quickly, typically in one second. Killed workers will not leave long-lived, orphaned locks to adversely block jobs from being enqueued.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/kinkade/active_job_resque_solo. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
License
The gem is available as open source under the terms of the MIT License.
Code of Conduct
Everyone interacting in the ActiveJobResqueSolo project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.