lib/lockfile.rb : a ruby library for creating perfect and NFS safe lockfiles

bin/rlock : ruby command line tool which uses this library to create lockfiles
            and to run arbitrary commands while holding them. 

            for example

              rlock lockfile -- cp -r huge/ huge.bak/

            run 'rlock -h' for more info


sudo ruby install.rb


* create a globally uniq filename in the same filesystem as the desired
  lockfile - this can be nfs mounted

* link(2) this file to the desired lockfile, ignore all errors

* stat the uniq filename and desired lockfile to determine is they are the
  same, use only stat.rdev and stat.ino - ignore stat.nlink as NFS can cause
  this report incorrect values 

* iff same, you have lock.  either return or run optional code block with
  optional refresher thread keeping lockfile fresh during execution of code
  block, ensuring that the lockfile is removed..

* iff not same try again a few times in rapid succession (poll), then, iff
  this fails, sleep using incremental backoff time.  optionally remove
  lockfile if it is older than a certain time, timeout if more than a certain
  amount of time has passed attempting to lock file.


  lockfile = Lockfile.new 'file.lock'
    p 42

  require 'pstore'           # which is NOT nfs safe on it's own

  opts = {                    # the keys can be symbols or strings

    :retries        => nil,   # we will try forever to aquire the lock

    :sleep_inc      => 2,     # we will sleep 2 seconds longer than the
                              # previous sleep after each retry, cycling from
                              # min_sleep upto max_sleep downto min_sleep upto
                              # max_sleep, etc., etc.

    :min_sleep      => 2,     # we will never sleep less than 2 seconds

    :max_sleep      => 32,    # we will never sleep longer than 32 seconds

    :max_age        => 3600,  # we will blow away any files found to be older
                              # than this (lockfile.thief? #=> true)

    :suspend        => 1800,  # iff we steal the lock from someone else - wait
                              # this long to give them a chance to realize it

    :refresh        => 8,     # we will spawn a bg thread that touches file
                              # every 8 sec.  this thread also causes a
                              # StolenLockError to be thrown if the lock
                              # disappears from under us - note that the
                              # 'detection' rate is limited to the refresh
                              # interval - this is a race condition

    :timeout        => nil,   # we will wait forever 

    :poll_retries   => 16,    # the initial attempt to grab a lock is done in a
                              # polling fashion, this number controls how many
                              # times this is done - the total polling attempts
                              # are considered ONE actual attempt (see retries
                              # above)

    :poll_max_sleep => 0.08,  # when polling a very brief sleep is issued
                              # between attempts, this is the upper limit of
                              # that sleep timeout 

    :dont_clean     => false, # normally a finalizer is defined to clean up
                              # after lockfiles, settin this to true prevents this 

    :dont_sweep     => false, # normally locking causes a sweep to be made.  a
                              # sweep removes any old tmp files created by
                              # processes of this host only which are no
                              # longer alive

    :debug => true,           # trace execution step on stdout

  pstore = PStore.new 'file.db'
  lockfile = Lockfile.new 'file.db.lock', opts 
  lockfile.lock do
    pstore.transaction do
      pstore[:last_update_time] = Time.now

3) same as 1 above - Lockfile.new takes a block and ensures unlock is called 

  Lockfile.new('file.lock') do
    p 42

4) watch locking algorithim in action (debugging only)

    Lockfile.debug = true
    Lockfile.new('file.lock') do
      p 42

  you can also set debugging via the ENV var LOCKFILE_DEBUG, eg.

  ~ > LOCKFILE_DEBUG=true rlock lockfile

5) simplified interface : no lockfile object required

    Lockfile('lock', :retries => 0) do
      puts 'only one instance running!'


* see samples/a.rb
* see samples/nfsstore.rb
* see samples/lock.sh
* see bin/rlock


Ara T. Howard




bugno > 1 && bugno < 42


  - lock fires up a refresher thread when called without a block

  - fixed a small non-critical bug in the require gaurd

  - upped defaults for max_age to 3600 and suspend to 1800.
  - tweaked a few things to play nice with rubygems

  - Mike Kasick <[email protected]> reported a bug whereby false/nil
    values for options were ignored.  patched to address this bug
  - added Lockfile method for high level interface sans lockfile object
  - updated rlock program to allow nil/true/false values passed on command
    line.  eg

      rlock --max_age=nil lockfile -- date --iso-8601=seconds

  - gem'd up
  - added Lockfile::create method which atomically creates a file and opens

      Lockfile::create("atomic_even_on_nfs", "r+") do |f|
        f.puts 42

    arguments are the same as those for File::open.  note that there is no way
    to accomplish this otherwise since File::O_EXCL fails silently on nfs,
    flock does not work on nfs, and posixlock (see raa) can only lock a file
    after it is open so the open itself is still a race.

  - added sweep functionality.  hosts can clean up any tmp files left around
    by other processes that may have been killed using -9 to prevent proper
    clean up.  it is only possible to clean up after processes on the same
    host as there is no other way to determine if the process that created the
    file is alive.  in summary - hosts clean up after other processes on that
    same host if needed.
  - added attempt/try_again methods
  - fixed bug where stale locks not removed when retries == 0
  - unfortunately i've forgotten
  - fixed bugette in sleep cycle where repeated locks with same lockfile would
    not reset the cycle at the start of each lock
  - allow rertries to be nil, meaning to try forever
  - default timeout is now nil - never timeout
  - default refresh is now 8
  - fixed bug where refresher thread was not actually touching lockfile! (ouch)
  - added cycle method to timeouts
    pattern is constructed using min_sleep, sleep_inc, max_sleep
  - removed use of yaml in favour of hand parsing the lockfile contents, the
    file as so small it just wasn't worth and i have had one core dump when yaml
    failed to parse a (corrupt) file
  - added an initial polling style attempt to grab lock before entering normal
    sleep/retry loop.  this has really helped performance when lock is under
    heavy contention: i see almost no sleeping done by any of in the interested
  - added ability of Lockfile.new to accept a block
  - initial version