Class: Sidekiq::JobRetry
- Inherits:
-
Object
- Object
- Sidekiq::JobRetry
- Includes:
- Component
- Defined in:
- lib/sidekiq/job_retry.rb
Overview
Automatically retry jobs that fail in Sidekiq. Sidekiq’s retry support assumes a typical development lifecycle:
0. Push some code changes with a bug in it.
1. Bug causes job processing to fail, Sidekiq's middleware captures
the job and pushes it onto a retry queue.
2. Sidekiq retries jobs in the retry queue multiple times with
an exponential delay, the job continues to fail.
3. After a few days, a developer deploys a fix. The job is
reprocessed successfully.
4. Once retries are exhausted, Sidekiq will give up and move the
job to the Dead Job Queue (aka morgue) where it must be dealt with
manually in the Web UI.
5. After 6 months on the DJQ, Sidekiq will discard the job.
A job looks like:
{ 'class' => 'HardJob', 'args' => [1, 2, 'foo'], 'retry' => true }
The ‘retry’ option also accepts a number (in place of ‘true’):
{ 'class' => 'HardJob', 'args' => [1, 2, 'foo'], 'retry' => 5 }
The job will be retried this number of times before giving up. (If simply ‘true’, Sidekiq retries 25 times)
Relevant options for job retries:
* 'queue' - the queue for the initial job
* 'retry_queue' - if job retries should be pushed to a different (e.g. lower priority) queue
* 'retry_count' - number of times we've retried so far.
* 'error_message' - the message from the exception
* 'error_class' - the exception class
* 'failed_at' - the first time it failed
* 'retried_at' - the last time it was retried
* 'backtrace' - the number of lines of error backtrace to store
We don’t store the backtrace by default as that can add a lot of overhead to the job and everyone is using an error service, right?
The default number of retries is 25 which works out to about 3 weeks You can change the default maximum number of retries in your initializer:
Sidekiq.default_configuration[:max_retries] = 7
or limit the number of retries for a particular job and send retries to a low priority queue with:
class MyJob
include Sidekiq::Job
retry: 10, retry_queue: 'low'
end
Defined Under Namespace
Constant Summary collapse
- DEFAULT_MAX_RETRY_ATTEMPTS =
25
Instance Attribute Summary
Attributes included from Component
Instance Method Summary collapse
-
#global(jobstr, queue) ⇒ Object
The global retry handler requires only the barest of data.
-
#initialize(capsule) ⇒ JobRetry
constructor
A new instance of JobRetry.
-
#local(jobinst, jobstr, queue) ⇒ Object
The local retry support means that any errors that occur within this block can be associated with the given job instance.
Methods included from Component
#fire_event, #handle_exception, #hostname, #identity, #inspect, #logger, #process_nonce, #redis, #safe_thread, #tid, #watchdog
Constructor Details
#initialize(capsule) ⇒ JobRetry
Returns a new instance of JobRetry.
75 76 77 78 79 |
# File 'lib/sidekiq/job_retry.rb', line 75 def initialize(capsule) @config = @capsule = capsule @max_retries = Sidekiq.default_configuration[:max_retries] || DEFAULT_MAX_RETRY_ATTEMPTS @backtrace_cleaner = Sidekiq.default_configuration[:backtrace_cleaner] end |
Instance Method Details
#global(jobstr, queue) ⇒ Object
The global retry handler requires only the barest of data. We want to be able to retry as much as possible so we don’t require the job to be instantiated.
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
# File 'lib/sidekiq/job_retry.rb', line 84 def global(jobstr, queue) yield rescue Handled => ex raise ex rescue Sidekiq::Shutdown => ey # ignore, will be pushed back onto queue during hard_shutdown raise ey rescue Exception => e # ignore, will be pushed back onto queue during hard_shutdown raise Sidekiq::Shutdown if exception_caused_by_shutdown?(e) msg = Sidekiq.load_json(jobstr) if msg["retry"] process_retry(nil, msg, queue, e) else @capsule.config.death_handlers.each do |handler| handler.call(msg, e) rescue => handler_ex handle_exception(handler_ex, {context: "Error calling death handler", job: msg}) end end raise Handled end |
#local(jobinst, jobstr, queue) ⇒ Object
The local retry support means that any errors that occur within this block can be associated with the given job instance. This is required to support the ‘sidekiq_retries_exhausted` block.
Note that any exception from the block is wrapped in the Skip exception so the global block does not reprocess the error. The Skip exception is unwrapped within Sidekiq::Processor#process before calling the handle_exception handlers.
117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
# File 'lib/sidekiq/job_retry.rb', line 117 def local(jobinst, jobstr, queue) yield rescue Handled => ex raise ex rescue Sidekiq::Shutdown => ey # ignore, will be pushed back onto queue during hard_shutdown raise ey rescue Exception => e # ignore, will be pushed back onto queue during hard_shutdown raise Sidekiq::Shutdown if exception_caused_by_shutdown?(e) msg = Sidekiq.load_json(jobstr) if msg["retry"].nil? msg["retry"] = jobinst.class.["retry"] end raise e unless msg["retry"] process_retry(jobinst, msg, queue, e) # We've handled this error associated with this job, don't # need to handle it at the global level raise Handled end |