Class: JobIteration::ThrottleEnumerator

Inherits:
Object
  • Object
show all
Defined in:
lib/job-iteration/throttle_enumerator.rb

Overview

ThrottleEnumerator allows you to throttle iterations based on external signal (e.g. database health). The enumerator from above will mimic +active_record_on_batches+, except when +DatabaseStatus.unhealthy?+ starts to return true. In that case, it will re-enqueue the job with a specified backoff.

Examples:

def build_enumerator(_params, cursor:)
  enumerator_builder.build_throttle_enumerator(
    enumerator_builder.active_record_on_batches(
      Account.inactive,
      cursor: cursor
    ),
    throttle_on: -> { DatabaseStatus.unhealthy? },
    backoff: 30.seconds
  )
end

Instance Method Summary collapse

Constructor Details

#initialize(enum, job, throttle_on:, backoff:) ⇒ ThrottleEnumerator

Returns a new instance of ThrottleEnumerator.



22
23
24
25
26
27
 
# File 'lib/job-iteration/throttle_enumerator.rb', line 22

def initialize(enum, job, throttle_on:, backoff:)
  @enum = enum
  @job = job
  @throttle_on = throttle_on
  @backoff = backoff
end

Instance Method Details

#should_throttle? ⇒ Boolean

Returns:

  • (Boolean) 


42
43
44
 
# File 'lib/job-iteration/throttle_enumerator.rb', line 42

def should_throttle?
  @throttle_on.call
end

#to_enum ⇒ Object 



29
30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/job-iteration/throttle_enumerator.rb', line 29

def to_enum
  Enumerator.new(-> { @enum.size }) do |yielder|
    @enum.each do |*val|
      if should_throttle?
        ActiveSupport::Notifications.instrument("throttled.iteration", job_class: @job.class.name)
        throw(:abort, [:retry, @backoff])
      end

      yielder.yield(*val)
    end
  end
end