BBQueue

BBQueue is an opinionated ruby gem to queue and process background jobs. Other gems for this purpose usually don't work with ruby objects and serialize method arguments only. Instead, BBQueue jobs are simple ruby objects:

MyQueue.enqueue MyJob.new

BBQueue jobs need to fulfill the following interface:

1. The object contains an instance method #work without any arguments
2. The object (instance) must be serializable via Marshal.dump and Marshal.load

Installation

Add this line to your application's Gemfile:

gem 'bbqueue'

And then execute:

$ bundle

Or install it yourself as:

$ gem install bbqueue

Usage

BBQueue is currently built on top of the stalking gem. Therefore, you have to install beanstalkd.

Producer

To enqueue a job, first create a queue, aka Producer:

SomeQueue = BBQueue::Producer.new("default")

where default is the queue name. You can pass stalking-specific options via:

AnotherQueue = BBQueue::Producer.new("another", :logger => ..., :delay => ..., :servers => ..., :ttr => ..., ...)

Then enqueue a job:

SomeQueue.enqueue MyJob.new("Some argument")

You can pass stalking-specific options here as well:

SomeQueue.enqueue MyJob.new("Some argument"), :delay => 5, ...

where MyJob looks like:

class MyJob
  attr_accessor :argument

  def initialize(argument)
    self.argument = argument
  end

  def work
    # Do some work
  end
end

Consumer

To process the enqueued jobs, create a file, e.g. jobs.rb:

BBQueue::Consumer.new "default"

and run it via:

$ bbqueue jobs.rb

BBQueue will loop through all the jobs, run them, and will then wait for new ones (no polling). You can pass multiple queue names and stalking-specific options:

BBQueue::Consumer.new ["default", "important"], :logger => ..., :servers => ...

By using a separate file like jobs.rb, you can load your environment or do other fancy things before finally calling BBQueue::Consumer.new. You can e.g. load your rails environment:

require File.expand_path("../environment", __FILE__)

Rails.application.eager_load!

SomeLogger.info "jobs booted into #{Rails.env.inspect} environment"

BBQueue::Consumer.new "background", :logger => SomeLogger

Forking Consumers

By default, BBQueue does not fork a process for every job, but it already ships with the ability to do so.

To make BBQueue fork, create a custom consumer:

class ForkingConsumer < BBQueue::Consumer
  def fork?
    true
  end

  def before_fork
    # ...
  end

  def after_fork
    # ...
  end
end

Then, start your custom consumer in favor of BBQueue's default one:

ForkingConsumer.new ["queue1", "queue2", ...], ...

Graceful Termination

Like for the stalking gem, you can stop a worker gracefully by sending a QUIT signal to it. The worker will finish its current job and terminate afterwards.

Contributing

1. Fork it ( https://github.com/mrkamel/bbqueue/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request