Module: Sidekiq::Throttled::Job::ClassMethods

Defined in:
lib/sidekiq/throttled/job.rb

Overview

Helper methods added to the singleton class of destination

Instance Method Summary collapse

Instance Method Details

#sidekiq_throttle(**kwargs) ⇒ void

This method returns an undefined value.

Registers some strategy for the worker.

Examples:

Allow max 123 MyJob jobs per hour


class MyJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle({
    :threshold => { :limit => 123, :period => 1.hour }
  })
end

Allow max 10 concurrently running MyJob jobs


class MyJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle({
    :concurrency => { :limit => 10 }
  })
end

Allow max 10 concurrent MyJob jobs and max 123 per hour


class MyJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle({
    :threshold => { :limit => 123, :period => 1.hour },
    :concurrency => { :limit => 10 }
  })
end

Allow max 123 MyJob jobs per hour; when jobs are throttled, schedule them for later in :other_queue


class MyJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle({
    :threshold => { :limit => 123, :period => 1.hour },
    :requeue => { :to => :other_queue, :with => :schedule }
  })
end

Parameters:

  • requeue (Hash) —

    What to do with jobs that are throttled

See Also:



93
94
95
96
97
98
99
100
101
102
 
# File 'lib/sidekiq/throttled/job.rb', line 93

def sidekiq_throttle(**kwargs)
  requeue_options = Throttled.config.default_requeue_options.merge(kwargs.delete(:requeue) || {})
  unless VALID_VALUES_FOR_REQUEUE_WITH.include?(requeue_options[:with])
    raise ArgumentError, "requeue: #{requeue_options[:with]} is not a valid value for :with"
  end

  self.sidekiq_throttled_requeue_options = requeue_options

  Registry.add(self, **kwargs)
end

#sidekiq_throttle_as(name) ⇒ void

This method returns an undefined value.

Adds current worker to preconfigured throttling strategy. Allows sharing same pool for multiple workers.

First of all we need to create shared throttling strategy:

# Create google_api throttling strategy
Sidekiq::Throttled::Registry.add(:google_api, {
  :threshold => { :limit => 123, :period => 1.hour },
  :concurrency => { :limit => 10 }
})

Now we can assign it to our workers:

class FetchProfileJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle_as :google_api
end

class FetchCommentsJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle_as :google_api
end

With the above configuration we ensure that there are maximum 10 concurrently running jobs of FetchProfileJob or FetchCommentsJob allowed. And only 123 jobs of those are executed per hour.

In other words, it will allow:

  • only ‘X` concurrent `FetchProfileJob`s

  • max ‘XX` `FetchProfileJob` per hour

  • only ‘Y` concurrent `FetchCommentsJob`s

  • max ‘YY` `FetchCommentsJob` per hour

Where ‘(X + Y) == 10` and `(XX + YY) == 123`

See Also:



146
147
148
 
# File 'lib/sidekiq/throttled/job.rb', line 146

def sidekiq_throttle_as(name)
  Registry.add_alias(self, name)
end