Postburner

An ActiveRecord layer on top of Backburner for inspecting and auditing the queue, especially for delayed jobs. It isn't meant to be outperform other queues, but be safe (and inspectable).

It is meant to be complementary to Backburner. Use Backburner as the default ActiveJob processor for mailers, active storage, and the like. Use a Postburner::Job for things that you want to track. See Comparison to Backburner for more.

Postburner meant to be a replacement/upgrade for Que. If you need something faster, check out Que - we love it! Postburner is built for a slightly different purpose: to be simple, safe, and inspectable. All of which Que has (or can be added), but are included by default with some architecture differences. See Comparison to Que for more.

Usage

class RunDonation < Postburner::Job
  queue 'critical'
  queue_priority 0 # 0 is highest priority
  queue_max_job_retries 0 # don't retry

  def perform(args)
    # do long tasks here
    # also have access to self.args
  end
end

# RunDonation#create! is the `ActiveRecord` method, so it returns a model,
# you can manipulate, add columns to the table to, reference with foreign
# keys, etc.
job = RunDonation.create!(args: {donation_id: 123})
# NOTE Make sure use use an after_commit or after_save_commit, etc to avoid
#      any race conditions of Postburner trying to process before a required
#      database mutation is commited.
job.queue!
=> {:status=>"INSERTED", :id=>"1139"}

# queue for later with `:at`
RunDonation.create!(args: {donation_id: 123}).queue! at: Time.zone.now + 2.days
=> {:status=>"INSERTED", :id=>"1140"}

# queue for later with `:delay`
#
# `:delay` takes priority over `:at`takes priority over `:at` because the
# beanstalkd protocol uses uses `delay`
RunDonation.create!(args: {donation_id: 123}).queue! delay: 1.hour
=> {:status=>"INSERTED", :id=>"1141"}

Mailers

j = Postburner::Mailer.
  delivery(UserMailer, :welcome)
  .with(name: 'Freddy')

j.queue!
=> {:status=>"INSERTED", :id=>"1139"}

Beaneater and beanstalkd attributes and methods

# get the beanstalkd job id
job.bkid
=> 1104

# get the beaneater job, call any beaneater methods on this object
job.beanstalk_job

# get the beanstald stats
job.beanstalk_job.stats

# kick the beanstalk job
job.kick!

# delete beankstalkd job, retain the job model
job.delete!

# delete beankstalkd job, retain the job model, but set `removed_at` on model.
job.remove!

# or simply remove the model, which will clean up the beanstalkd job in a before_destroy hook
job.destroy # OR job.destroy!

# get a cached Backburner connection and inspect it (or even use it directly)
c = Postburner.connection
c.beanstalk.tubes.to_a
c.beanstalk.tubes.to_a.map{|t| c.tubes[t.name].peek(:buried)}
c.beanstalk.tubes['ss.development.caching'].stats
c.beanstalk.tubes['ss.development.caching'].peek(:buried).kick
c.beanstalk.tubes['ss.development.caching'].kick(3)
c.close

# automatically close
Postburner.connected do |connection|
  # do stuff with connection
end

Read about the beanstalkd protocol.

Basic model fields

# ActiveRecord primary key
job.id

# int id of the beankstalkd job
job.bkid

# string uuid
job.sid

# ActiveRecord STI job subtype
job.type

# jsonb arguments for use in job
job.args

# time job should run - not intended to be changed
# TODO run_at should be readonly after create
job.run_at

Job statistics

# when job was inserted into beankstalkd
job.queued_at

# last time attempted
job.attempting_at

# last time processing started
job.processing_at

# when completed
job.processed_at

# when removed, may be nil
job.removed_at

# lag in ms from run_at/queued_at to attempting_at
job.lag

# duration of processing in ms
job.duration

# number of attempts
job.attempt_count

# number of errors (length of errata)
job.error_count

# number of log entries (length of logs)
job.log_count

# array of attempting_at times
job.attempts

# array of errors
job.errata

# array of log messages
job.logs

Job logging and exceptions

Optionally, you can:

  1. Add log messages to the job during processing to logs
  2. Add log your own exceptions to errata
class RunDonation < Postburner::Job
  queue 'critical'
  queue_priority 0 # 0 is highest priority
  queue_max_job_retries 0 # don't retry

  def perform(args)
    # log at task, defaults to `:info`, but `:debug`, `:warning`, `:error`
    log "Log bad condition...", level: :error

    begin
      # danger
    rescue Exception => e
      log_exception e
    end
  end
end

Optionally, mount the engine

mount Postburner::Engine => "/postburner"

# mount only for development inspection
mount Postburner::Engine => "/postburner" if Rails.env.development?

Open the controller to add your own authentication or changes - or just create your own routes, controllers, and views.

Override the views to make them prettier - or follow the suggestion above and use your own.

Known Issues

  1. Beaneater and/or Beanstalkd seems to transform a tube name with hyphens to underscores upon instrospection of stats. See example. Reccommend using names without hyphens / dashes.
(main):001:0> Postburner.connection.tubes["backburner.worker.queue.cat-dog"].put("{}", delay: 1.week.to_i)
=> {:status=>"INSERTED", :id=>"9"}
irb(main):002:0> Postburner.connection.jobs.find(9)
=> #<Beaneater::Job id=9 body="{}">
irb(main):003:0> Postburner.connection.jobs.find(9).stats
=> #<Beaneater::StatStruct id=9, tube="backburner.worker.queue.cat_dog", state="delayed", pri=65536, age=23, delay=604800, ttr=120, time_left=604776, file=0, reserves=0, timeouts=0, releases=0, buries=0, kicks=0>
#
# NOTE 'cat-dog' on line 1 and 'cat_dog' on line 3.
#

Installation

First install beanstalkd. On Debian-based systems, that goes like this:

sudo apt-get install beanstalkd

Then add to your Gemfile.

gem 'postburner'

Install...

$ bundle

After bundling, install the migration.

# install migration, possible to edit and add attributes or indexes as needed.
$ bundle exec rails g postburner:install

Inspect XXX_create_postburner_jobs.rb. (The template is here).The required attributes are in there, but add more if you would like to use them, or do it in a separate migration. By default, several indexes are added

Because Postburner is built on Backburner, add a config/initializers/backburner.rb with option found here.

Backburner.configure do |config|
  config.beanstalk_url       = "beanstalk://127.0.0.1"
  config.tube_namespace      = "some.app.production"
  config.namespace_separator = "."
  config.on_error            = lambda { |e| puts e }
  config.max_job_retries     = 3 # default 0 retries
  config.retry_delay         = 2 # default 5 seconds
  config.retry_delay_proc    = lambda { |min_retry_delay, num_retries| min_retry_delay + (num_retries ** 3) }
  config.default_priority    = 65536
  config.respond_timeout     = 120
  config.default_worker      = Backburner::Workers::Simple
  config.logger              = Logger.new(STDOUT)
  config.primary_queue       = "backburner-jobs"
  config.priority_labels     = { :custom => 50, :useless => 1000 }
  config.reserve_timeout     = nil
  config.job_serializer_proc = lambda { |body| JSON.dump(body) }
  config.job_parser_proc     = lambda { |body| JSON.parse(body) }
end

Finally, set Backburner for ActiveJob

# config/application.rb
config.active_job.queue_adapter = :backburner

Postburner may later provide an adapter, but we recommend using Postburner::Job classes directly.

Add jobs to app/jobs/. There currently is no generator.

# app/jobs/run_donation.rb
class RunDonation < Postburner::Job
  queue 'critical'
  queue_priority 0 # 0 is highest priority
  queue_max_job_retries 0 # don't retry

  def process(args)
    # do long tasks here
    # also have access to self.args
  end
end

Comparison to Backburner

Compared to plain Backburner, Postburner adds:

  • Database Jobs for inspection, linking, auditing, removal (and deletion)
  • Direct access to associated Beanstalk (via beaneater)
  • Job Statistics (lag, attempts, logs, tracked errors)
  • Convenience methods to clear tubes, stats, and connections for Beanstalk.

Otherwise, Postburner tries to be a super simple layer on Backburner::Queue and ActiveRecord. Every tool with either of those are available in Postburner::Jobs.

Comes with a mountable interface that can be password protected with whatever authentication you use in your project.

Comparison to Que

Postburner meant to be a replacement/upgrade for Que. However, if you need something faster and backed with ACID compliance, check out Que.

Postburner has some additional features such as retained jobs after processing, stats, per job logging, etc.

Postburner is meant to be simpler than Que. Que is incredible, but jobs should be simple so the logic and history can be transparent.

Contributing

Submit a pull request. Follow conventions of the project. Be nice.

V1 TODO

  • Basic tests
  • Add Authentication modules for engine mount.

V1+ TODO

  • Install generator
    • Sub to backburner
  • Job generator
    • Build file in app/jobs
    • Inherit from Postburner::Job
  • Add before/after/around hooks
  • Add destroy, and remove actions on show page
  • Clear tubes.
  • Document how/when to use activerecord hooks
  • Document how/when to use backburner hooks
  • Document how/when to use postburner hooks
  • Add logging with Job.args in backburner logs
  • MAYBE - ActiveJob adapter

Running in Development

cd test/dummy
bundle exec backburner

Releasing

  1. gem bump -v minor -t Where can be: major|minor|patch|pre|release or a version number
  2. Edit CHANGELOG.md with details from git log --oneline
  3. git commit --amend
  4. gem release -k nac -p Where is an authorized key with push capabilities.

License

The gem is available as open source under the terms of the MIT License.