Class: ProgressLogger

Inherits:
Object
  • Object
show all
Defined in:
lib/progress-logger.rb

Overview

The ProgressLogger class is the workhorse of this gem - it is used to wrap your logging code (or whatever you are doing) you construct it with a set of criteria for when it should log, and a block to do the actual logging (or other activity): <tt>

p = ProgressLogger.new(:step => 100000) do |state|
   puts "processed #{state.count} rows"
end

</tt> and then every time “p.trigger()” is called:

  • p.count is incremented

  • if p.count is a multiple of 100000 the block is called, with a ‘state’ object as a parameter

You can do pretty well anything you want in the passed block - log something, flush a database, update a gui, whatever.

See the ProgressLogger::State class for what you can get from the state object

Examples

Log every 5 minutes:

<tt>

p = ProgressLogger.new(:minutes => 5) do |state|
   puts "processed #{state.count} rows"
end

</tt>

Log every hour, or after a million triggers, with a detailed message and some extra work

<tt>

max = @collection.size
p = ProgressLogger.new(:hours => 1, :step => 1000000, :max => max) do |state|
   @logger.info "processed #{state.count} rows"
   @logger.info "  Current processing rate of #{state.short_rate} rows/sec implies ending in #{state.short_eta/(3600)} hours"
   @logger.info "  Long-term processing rate of #{state.long_rate} rows/sec implies ending in #{state.long_eta/(3600)} hours"
   @logger.debug "flushing cache"
   @cache.flush
end
@collection.find().each do |row|
  p.trigger
  process(row)
end
@logger.info "done - processed #{p.count} rows in total"

</tt>

Passing the ProgressLogger around

<tt>

  parent_count = 0
  plogger = ProgressLogger.new(:minutes => 5) do |state|
     puts "processed #{parent_count} parents, #{state.count} children"
  end
  @parents.each do |parent|
    parent_count += 1
    process_children(parent, plogger)
  end
end
def process_children(parent, plogger)
  parent.children.each do |child|
    plogger.trigger
    ... do stuff
  end
end

</tt>

Defined Under Namespace

Classes: State

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(params = {}, &block) ⇒ ProgressLogger

create a ProgressLogger with specified criteria you must specify either a :step or one of :seconds, :minutes, and/or :hours parameters:

  • :step - the passed block is called after this many calls to trigger()

  • :seconds, :minutes, :hours - the passed block is called after this number of seconds/minutes/hours

    • you can specify more than one of these, they’ll just get added together

  • :max - this is an expected maximum number of triggers - it’s used to calculate eta values for the ProgressLogger::State object

  • block - you must pass a block, it is called (with a state parameter) when the above criteria are met

Raises:

  • (ArgumentError) 


139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/progress-logger.rb', line 139

def initialize(params = {}, &block)
  unless params[:step] || params[:seconds] || params[:minutes] || params[:hours]
    raise ArgumentError.new("You must specify a :step, :seconds, :minutes or :hours interval criterion to ProgressLogger")
  end
  unless block_given?
    raise ArgumentError.new("You must pass a block to ProgressLogger")
  end
  @stepsize = params[:step]
  raise ArgumentError.new("Step size must be greater than 0") if @stepsize && @stepsize <= 0
  @max = params[:max]
  raise ArgumentError.new("Max count must be greater than 0") if @max && @max <= 0

  @count_based = params[:step]
  @time_based = params[:seconds] || params[:minutes] || params[:hours]
  if @time_based
    @seconds = params[:seconds] || 0
    @seconds += params[:minutes] * 60 if params[:minutes]
    @seconds += params[:hours] * 60 * 60 if params[:hours]
    raise ArgumentError.new("You must specify a total time greater than 0") if @seconds <= 0
  end

  @count = 0
  @block = block
  @started = false  # don't start yet - allow for startup time in loops; can start manually with start() below
end

Instance Attribute Details

#count ⇒ Object (readonly)

count of triggers processed so far



128
129
130
 
# File 'lib/progress-logger.rb', line 128

def count
  @count
end

Instance Method Details

#start(now = Time.now) ⇒ Object

manually start timers normally timers are initialized on the first call to trigger() - this is because quite often, a processing loop like the following has a big startup time as cursors are allocated etc: <tt>

p = ProgressLogger.new ...
@db.find({:widget => true).each do  # this takes 5 minutes to cache cursors!
  p.trigger

</tt> If the timers were initialized when ProgressLogger.new was called, they’d be messed up by the loop start time. If for some reason you want the timers to be manually started earlier, you can explicitly call start, optionally passing it your own special version of Time.now



176
177
178
179
180
181
182
 
# File 'lib/progress-logger.rb', line 176

def start(now = Time.now)
  @started = true
  @start_time = now
  @last_report = now
  @last_timecheck = now  # last time interval, so count-based reports don't stop time-based reports
  @start_count = @last_count = @count
end

#trigger ⇒ Object

trigger whatever regular event you are watching - if the criteria are met, this will call your block of code



185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
 
# File 'lib/progress-logger.rb', line 185

def trigger
  start unless @started  # note - will set start and last counts to 0, which may be slightly inaccurate!
  @count += 1
  now = Time.now
  if @time_based
    time_delta = now - @last_timecheck
    its_time = (time_delta > @seconds)
  else
    its_time = false
  end
  its_enough = @count_based && (@count % @stepsize == 0)
  if its_time || its_enough
    run_block now
    @last_report = now
    @last_count = @count
    @last_timecheck = now if its_time
  end
end