Throttler
Throttler rate-limits code execution across threads, evented blocks, or processes on a server.
Installation
Throttler works only on platforms that support file locking.
# Gemfile
gem 'throttler'
Usage
The following background job ensures workers will not scrape a site faster than once every second per IP address.
class Scrape
def self.perform(site, ip_address, *ids)
Throttler.limit 1.0, site, ip_address do
spider = Spider.new site, ip_address
spider.scrape *ids
end
end
end