progress-logger

This is a very simple gem to make it easier to log the state of long slow processing jobs. I tend to do this sort of thing over and over and over:

count = 0
@collection.each do |record|
  count += 1
  if count % 1000000 == 0
    puts "Processed #{count} rows"
  end
  # do stuff
end

with variations for timed reporting, calculating processing rate, and so on. ProgressLogger really doesn’t do much - it just tries to handle the interval checking, leaving what you do every million records (or whatever) entirely up to you. The example above can be run as:

p = ProgressLogger.new(:count => 1000000) do |state|
  puts "Processed #{state.count} rows"
end
@collection.each do |record|
  p.trigger
  # do stuff
end

You can do lots of other stuff too - see the ProgressLogger rdocs for more, but an example to whet your appetite:

p = ProgressLogger.new(:count => 1000000, :minutes => 30, :max => @collection.size) do |state|
  @logger.info("Processed #{state.count} rows - #{state.long_eta/(60*60)} hours to go!")
  @cache.flush()
end

More examples are in the ProgressLogger docs - or take a look at the specs!

Note on Patches/Pull Requests

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.

progress-logger

Note on Patches/Pull Requests

Copyright