Module: Resque::Plugins::Retry

Included in:
ExponentialBackoff
Defined in:
lib/resque/plugins/retry.rb

Overview

If you want your job to retry on failure, simply extend your module/class with this module:

class DeliverWebHook
  extend Resque::Plugins::Retry # allows 1 retry by default.
  @queue = :web_hooks

  def self.perform(url, hook_id, hmac_key)
    heavy_lifting
  end
end

Easily do something custom:

class DeliverWebHook
  extend Resque::Plugins::Retry
  @queue = :web_hooks

  @retry_limit = 8  # default: 1
  @retry_delay = 60 # default: 0

  # used to build redis key, for counting job attempts.
  def self.identifier(url, hook_id, hmac_key)
    "#{url}-#{hook_id}"
  end

  def self.perform(url, hook_id, hmac_key)
    heavy_lifting
  end
end

Instance Method Summary collapse

Instance Method Details

#after_perform_retry(*args) ⇒ Object

Resque after_perform hook.

Deletes retry attempt count from Redis.



209
210
211
 
# File 'lib/resque/plugins/retry.rb', line 209

def after_perform_retry(*args)
  Resque.redis.del(redis_retry_key(*args))
end

#args_for_retry(*args) ⇒ Array

This method is abstract.

Modify the arguments used to retry the job. Use this to do something other than try the exact same job again.

Returns:

  • (Array) —

    new job arguments



96
97
98
 
# File 'lib/resque/plugins/retry.rb', line 96

def args_for_retry(*args)
  args
end

#before_perform_retry(*args) ⇒ Object

Resque before_perform hook.

Increments and sets the ‘@retry_attempt` count.



200
201
202
203
204
 
# File 'lib/resque/plugins/retry.rb', line 200

def before_perform_retry(*args)
  retry_key = redis_retry_key(*args)
  Resque.redis.setnx(retry_key, -1)             # default to -1 if not set.
  @retry_attempt = Resque.redis.incr(retry_key) # increment by 1.
end

#clean_retry_key(*args) ⇒ Object 



193
194
195
 
# File 'lib/resque/plugins/retry.rb', line 193

def clean_retry_key(*args)
  Resque.redis.del(redis_retry_key(*args))
end

#identifier(*args) ⇒ String

This method is abstract.

You may override to implement a custom identifier, you should consider doing this if your job arguments are many/long or may not cleanly cleanly to strings.

Builds an identifier using the job arguments. This identifier is used as part of the redis key.

Parameters:

  • args (Array) —

    job arguments

Returns:

  • (String) —

    job identifier



52
53
54
55
 
# File 'lib/resque/plugins/retry.rb', line 52

def identifier(*args)
  args_string = args.join('-')
  args_string.empty? ? nil : args_string
end

#inherited(subclass) ⇒ Object

Copy retry criteria checks on inheritance.



38
39
40
41
 
# File 'lib/resque/plugins/retry.rb', line 38

def inherited(subclass)
  super(subclass)
  subclass.instance_variable_set("@retry_criteria_checks", retry_criteria_checks.dup)
end

#instance_exec(*args, &block) ⇒ Object 



225
226
227
228
229
230
231
232
233
234
 
# File 'lib/resque/plugins/retry.rb', line 225

def instance_exec(*args, &block)
  mname = "__instance_exec_#{Thread.current.object_id.abs}"
  class << self; self end.class_eval{ define_method(mname, &block) }
  begin
    ret = send(mname, *args)
  ensure
    class << self; self end.class_eval{ undef_method(mname) } rescue nil
  end
  ret
end

#on_failure_retry(exception, *args) ⇒ Object

Resque on_failure hook.

Checks if our retry criteria is valid, if it is we try again. Otherwise the retry attempt count is deleted from Redis.



217
218
219
220
221
222
223
 
# File 'lib/resque/plugins/retry.rb', line 217

def on_failure_retry(exception, *args)
  if retry_criteria_valid?(exception, *args)
    try_again(*args)
  else
    Resque.redis.del(redis_retry_key(*args))
  end
end

#redis_retry_key(*args) ⇒ String

Builds the redis key to be used for keeping state of the job attempts.

Returns:

  • (String) —

    redis key



61
62
63
 
# File 'lib/resque/plugins/retry.rb', line 61

def redis_retry_key(*args)
  ['resque-retry', name, identifier(*args)].compact.join(":").gsub(/\s/, '')
end

#retry_attempt ⇒ Fixnum

Number of retry attempts used to try and perform the job.

The real value is kept in Redis, it is accessed and incremented using a before_perform hook.

Returns:

  • (Fixnum) —

    number of attempts



79
80
81
 
# File 'lib/resque/plugins/retry.rb', line 79

def retry_attempt
  @retry_attempt ||= 0
end

#retry_criteria_check {|exception, *args| ... } ⇒ Object

Register a retry criteria check callback to be run before retrying the job again.

If any callback returns ‘true`, the job will be retried.

Examples:

Using a custom retry criteria check.


retry_criteria_check do |exception, *args|
  if exception.message =~ /InvalidJobId/
    # don't retry if we got passed a invalid job id.
    false
  else
    true
  end
end

Yields:

  • (exception, *args)

Yield Parameters:

  • exception (Exception) —

    the exception that was raised

  • args (Array) —

    job arguments

Yield Returns:

  • (Boolean) —

    false == dont retry, true = can retry



176
177
178
 
# File 'lib/resque/plugins/retry.rb', line 176

def retry_criteria_check(&block)
  retry_criteria_checks << block
end

#retry_criteria_checks ⇒ Array

Retry criteria checks.

Returns:

  • (Array) 


141
142
143
144
 
# File 'lib/resque/plugins/retry.rb', line 141

def retry_criteria_checks
  @retry_criteria_checks ||= []
  @retry_criteria_checks
end

#retry_criteria_valid?(exception, *args) ⇒ Boolean

Test if the retry criteria is valid.

Parameters:

  • exception (Exception)
  • args (Array) —

    job arguments

Returns:

  • (Boolean) 


123
124
125
126
127
128
129
130
131
132
133
134
135
136
 
# File 'lib/resque/plugins/retry.rb', line 123

def retry_criteria_valid?(exception, *args)
  # if the retry limit was reached, dont bother checking anything else.
  return false if retry_limit_reached?

  # We always want to retry if the exception matches.
  should_retry = retry_exception?(exception.class)

  # call user retry criteria check blocks.
  retry_criteria_checks.each do |criteria_check|
    should_retry ||= !!instance_exec(exception, *args, &criteria_check)
  end

  should_retry
end

#retry_delay ⇒ Number

This method is abstract.

Number of seconds to delay until the job is retried.

Returns:

  • (Number) —

    number of seconds to delay



87
88
89
 
# File 'lib/resque/plugins/retry.rb', line 87

def retry_delay
  @retry_delay ||= 0
end

#retry_exception?(exception) ⇒ Boolean

Convenience method to test whether you may retry on a given exception.

Returns:

  • (Boolean) 


103
104
105
106
 
# File 'lib/resque/plugins/retry.rb', line 103

def retry_exception?(exception)
  return true if retry_exceptions.nil?
  !! retry_exceptions.any? { |ex| ex >= exception }
end

#retry_exceptions ⇒ Array?

This method is abstract.

Controls what exceptions may be retried.

Default: ‘nil` - this will retry all exceptions.

Returns:

  • (Array, nil) 


114
115
116
 
# File 'lib/resque/plugins/retry.rb', line 114

def retry_exceptions
  @retry_exceptions ||= nil
end

#retry_limit ⇒ Fixnum

Maximum number of retrys we can attempt to successfully perform the job. A retry limit of 0 or below will retry forever.

Returns:

  • (Fixnum) 


69
70
71
 
# File 'lib/resque/plugins/retry.rb', line 69

def retry_limit
  @retry_limit ||= 1
end

#retry_limit_reached? ⇒ Boolean

Test if the retry limit has been reached.

Returns:

  • (Boolean) 


149
150
151
152
153
154
 
# File 'lib/resque/plugins/retry.rb', line 149

def retry_limit_reached?
  if retry_limit > 0
    return true if retry_attempt >= retry_limit
  end
  false
end

#try_again(*args) ⇒ Object

Will retry the job.



181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/resque/plugins/retry.rb', line 181

def try_again(*args)
  retry_in_queue = @retry_job_class ? @retry_job_class : self
  if retry_delay <= 0
    # If the delay is 0, no point passing it through the scheduler
    Resque.enqueue(retry_in_queue, *args_for_retry(*args))
  else
    Resque.enqueue_in(retry_delay, retry_in_queue, *args_for_retry(*args))
  end

  clean_retry_key(*args) if @retry_job_class
end