Class: Autotest

Inherits:
Object
  • Object
show all
Defined in:
lib/autotest.rb

Overview

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

  • Test files must be stored in test/

  • Test files names must start with test_

  • Test class names must start with Test

  • Implementation files must be stored in lib/

  • Implementation files must match up with a test file named test_.*implementation.rb

Strategy:

  1. Find all files and associate them from impl <-> test.

  2. Run all tests.

  3. Scan for failures.

  4. Detect changes in ANY (ruby?. file, rerun all failures + changed files.

  5. Until 0 defects, goto 3.

  6. When 0 defects, goto 2.

Defined Under Namespace

Modules: AutoUpdate, Once, RCov, Restart, Timestamp

Constant Summary collapse

VERSION = 
File.read( File.join(File.dirname(__FILE__),'..','VERSION') ).strip
T0 = 
Time.at 0
ALL_HOOKS = 
[ :all_good, :died, :green, :initialize, :interrupt, :quit,
:ran_command, :red, :reset, :run_command, :updated, :waiting ]
HOOKS =

unfound keys are []

Hash.new { |h,k| h[k] = [] }
WINDOZE = 
/win32/ =~ RUBY_PLATFORM
SEP = 
WINDOZE ? '&' : ';'
@@options = 
{}
@@discoveries = 
[]

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Autotest

Initialize the instance and then load the user’s .autotest file, if any.



154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
 
# File 'lib/autotest.rb', line 154

def initialize
  # these two are set directly because they're wrapped with
  # add/remove/clear accessor methods
  @exception_list = []
  @test_mappings = []

  self.completed_re = /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/
  self.extra_class_map   = {}
  self.extra_files       = []
  self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
  self.files_to_test     = new_hash_of_arrays
  self.find_order        = []
  self.known_files       = nil
  self.libs              = %w[. lib test].join(File::PATH_SEPARATOR)
  self.order             = :random
  self.output            = $stderr
  self.sleep             = 1
  self.testlib           = "test/unit"
  self.find_directories  = ['.']
  self.unit_diff         = "#{File.expand_path("#{File.dirname(__FILE__)}/../bin/unit_diff")} -u"

  #add Test::Unit mappings
  #file in /lib -> run test in /test
  self.add_mapping(/^lib\/.*\.rb$/) do |filename, _|
    possible = File.basename(filename).gsub '_', '_?'
    files_matching %r%^test/.*#{possible}$%
  end

  #file in /test -> run it
  self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _|
    filename
  end

  #execute custom extensions
  load_custom_extensions(options[:rc])
end

Instance Attribute Details

#completed_re ⇒ Object

Returns the value of attribute completed_re.



132
133
134
 
# File 'lib/autotest.rb', line 132

def completed_re
  @completed_re
end

#extra_class_map ⇒ Object

Returns the value of attribute extra_class_map.



132
133
134
 
# File 'lib/autotest.rb', line 132

def extra_class_map
  @extra_class_map
end

#extra_files ⇒ Object

Returns the value of attribute extra_files.



132
133
134
 
# File 'lib/autotest.rb', line 132

def extra_files
  @extra_files
end

#failed_results_re ⇒ Object

Returns the value of attribute failed_results_re.



132
133
134
 
# File 'lib/autotest.rb', line 132

def failed_results_re
  @failed_results_re
end

#files_to_test ⇒ Object

Returns the value of attribute files_to_test.



132
133
134
 
# File 'lib/autotest.rb', line 132

def files_to_test
  @files_to_test
end

#find_directories ⇒ Object

Returns the value of attribute find_directories.



132
133
134
 
# File 'lib/autotest.rb', line 132

def find_directories
  @find_directories
end

#find_order ⇒ Object

Returns the value of attribute find_order.



132
133
134
 
# File 'lib/autotest.rb', line 132

def find_order
  @find_order
end

#interrupted ⇒ Object

Returns the value of attribute interrupted.



132
133
134
 
# File 'lib/autotest.rb', line 132

def interrupted
  @interrupted
end

#known_files ⇒ Object

Lazy accessor for the known_files hash.



424
425
426
427
428
429
 
# File 'lib/autotest.rb', line 424

def known_files
  unless @known_files then
    @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
  end
  @known_files
end

#last_mtime ⇒ Object

Returns the value of attribute last_mtime.



132
133
134
 
# File 'lib/autotest.rb', line 132

def last_mtime
  @last_mtime
end

#libs ⇒ Object

Returns the value of attribute libs.



132
133
134
 
# File 'lib/autotest.rb', line 132

def libs
  @libs
end

#order ⇒ Object

Returns the value of attribute order.



132
133
134
 
# File 'lib/autotest.rb', line 132

def order
  @order
end

#output ⇒ Object

Returns the value of attribute output.



132
133
134
 
# File 'lib/autotest.rb', line 132

def output
  @output
end

#results ⇒ Object

Returns the value of attribute results.



132
133
134
 
# File 'lib/autotest.rb', line 132

def results
  @results
end

#sleep ⇒ Object

Returns the value of attribute sleep.



132
133
134
 
# File 'lib/autotest.rb', line 132

def sleep
  @sleep
end

#tainted ⇒ Object

Returns the value of attribute tainted.



132
133
134
 
# File 'lib/autotest.rb', line 132

def tainted
  @tainted
end

#testlib ⇒ Object

Returns the value of attribute testlib.



132
133
134
 
# File 'lib/autotest.rb', line 132

def testlib
  @testlib
end

#unit_diff ⇒ Object

Returns the value of attribute unit_diff.



132
133
134
 
# File 'lib/autotest.rb', line 132

def unit_diff
  @unit_diff
end

#wants_to_quit ⇒ Object

Returns the value of attribute wants_to_quit.



132
133
134
 
# File 'lib/autotest.rb', line 132

def wants_to_quit
  @wants_to_quit
end

Class Method Details

.add_discovery(&proc) ⇒ Object

Add a proc to the collection of discovery procs. See autodiscover.



85
86
87
 
# File 'lib/autotest.rb', line 85

def self.add_discovery &proc
  @@discoveries << proc
end

.add_hook(name, &block) ⇒ Object

Add the supplied block to the available hooks, with the given name.



665
666
667
 
# File 'lib/autotest.rb', line 665

def self.add_hook(name, &block)
  HOOKS[name] << block
end

.autodiscover ⇒ Object

Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb”. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.

Process:

  1. All autotest/discover.rb files loaded.

  2. Those procs determine your styles (eg [“rails”, “rspec”]).

  3. Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).

  4. Invoke run method on appropriate class (eg Autotest::RailsRspec.run).

Example autotest/discover.rb:

Autotest.add_discovery do
  "rails" if File.exist? 'config/environment.rb'
end



113
114
115
116
117
118
119
120
121
122
 
# File 'lib/autotest.rb', line 113

def self.autodiscover
  require 'rubygems'

  Gem.find_files("autotest/discover").each do |f|
    load f
  end

  #call all discover procs an determine style
  @@discoveries.map{ |proc| proc.call }.flatten.compact.sort.uniq
end

.options ⇒ Object 



69
 
# File 'lib/autotest.rb', line 69

def self.options;@@options;end

.run ⇒ Object

Initialize and run the system.



127
128
129
 
# File 'lib/autotest.rb', line 127

def self.run
  new.run
end

Instance Method Details

#add_exception(regexp) ⇒ Object

Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



596
597
598
599
600
 
# File 'lib/autotest.rb', line 596

def add_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list << regexp
  nil
end

#add_mapping(regexp, prepend = false, &proc) ⇒ Object

Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.

For example, if test_helper.rb is modified, rerun all tests:

at.add_mapping(/test_helper.rb/) do |f, _|
  at.files_matching(/^test.*rb$/)
end



560
561
562
563
564
565
566
567
 
# File 'lib/autotest.rb', line 560

def add_mapping(regexp, prepend = false, &proc)
  if prepend then
    @test_mappings.unshift [regexp, proc]
  else
    @test_mappings.push [regexp, proc]
  end
  nil
end

#add_sigint_handler ⇒ Object

Installs a sigint handler.



293
294
295
296
297
298
299
300
301
302
303
304
305
306
 
# File 'lib/autotest.rb', line 293

def add_sigint_handler
  trap 'INT' do
    if self.interrupted then
      self.wants_to_quit = true
    else
      unless hook :interrupt then
        puts "Interrupt a second time to quit"
        self.interrupted = true
        Kernel.sleep 1.5
      end
      raise Interrupt, nil # let the run loop catch it
    end
  end
end

#all_good ⇒ Object

If there are no files left to test (because they’ve all passed), then all is good.



312
313
314
 
# File 'lib/autotest.rb', line 312

def all_good
  files_to_test.empty?
end

#clear_exceptions ⇒ Object

Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.



616
617
618
619
620
 
# File 'lib/autotest.rb', line 616

def clear_exceptions
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.clear
  nil
end

#clear_mappings ⇒ Object

Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.



584
585
586
587
 
# File 'lib/autotest.rb', line 584

def clear_mappings
  @test_mappings.clear
  nil
end

#consolidate_failures(failed) ⇒ Object

Returns a hash mapping a file name to the known failures for that file.



332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/autotest.rb', line 332

def consolidate_failures(failed)
  filters = new_hash_of_arrays

  class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly
                     [path_to_classname(f), f]
                   }.flatten]
  class_map.merge!(self.extra_class_map)

  failed.each do |method, klass|
    if class_map.has_key? klass then
      filters[class_map[klass]] << method
    else
      output.puts "Unable to map class #{klass} to a file"
    end
  end

  return filters
end

#exceptions ⇒ Object

Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.



627
628
629
630
631
632
633
634
635
636
637
 
# File 'lib/autotest.rb', line 627

def exceptions
  unless defined? @exceptions then
    if @exception_list.empty? then
      @exceptions = nil
    else
      @exceptions = Regexp.union(*@exception_list)
    end
  end

  @exceptions
end

#files_matching(regexp) ⇒ Object

Returns all known files in the codebase matching regexp.



544
545
546
 
# File 'lib/autotest.rb', line 544

def files_matching regexp
  self.find_order.select { |k| k =~ regexp }
end

#find_files ⇒ Object

Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.



356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
 
# File 'lib/autotest.rb', line 356

def find_files
  result = {}
  targets = self.find_directories + self.extra_files
  self.find_order.clear

  targets.each do |target|
    order = []
    Find.find(target) do |f|
      Find.prune if f =~ self.exceptions

      next if test ?d, f
      next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files
      next if f =~ /\/\.?#/            # Emacs autosave/cvs merge files

      filename = f.sub(/^\.\//, '')

      result[filename] = File.stat(filename).mtime rescue next
      order << filename
    end
    self.find_order.push(*order.sort)
  end

  return result
end

#find_files_to_test(files = find_files) ⇒ Object

Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns the latest mtime of the files modified or nil when nothing was modified.



386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
 
# File 'lib/autotest.rb', line 386

def find_files_to_test(files=find_files)
  updated = files.select { |filename, mtime| self.last_mtime < mtime }

  unless updated.empty? or self.last_mtime.to_i == 0 #nothing to update or initial run
    p updated if options[:verbose]
    hook :updated, updated
  end

  updated.map { |f,m| test_files_for(f) }.flatten.uniq.each do |filename|
    self.files_to_test[filename] # creates key with default value
  end

  if updated.empty? then
    nil
  else
    files.values.max
  end
end

#get_to_green ⇒ Object

Keep running the tests after a change, until all pass.



236
237
238
239
240
241
 
# File 'lib/autotest.rb', line 236

def get_to_green
  begin
    run_tests
    wait_for_changes unless all_good
  end until all_good
end

#handle_results(results) ⇒ Object

Check results for failures, set the “bar” to red or green, and if there are failures record this.



409
410
411
412
413
414
415
416
417
418
419
 
# File 'lib/autotest.rb', line 409

def handle_results(results)
  failed = results.scan(self.failed_results_re)
  completed = results =~ self.completed_re

  self.files_to_test = consolidate_failures failed if completed

  color = completed && self.files_to_test.empty? ? :green : :red
  hook color unless $TESTING

  self.tainted = true unless self.files_to_test.empty?
end

#hook(name, *args) ⇒ Object

Call the event hook named name, executing all registered hooks until one returns true. Returns false if no hook handled the event.



647
648
649
650
651
652
653
654
655
656
657
658
659
 
# File 'lib/autotest.rb', line 647

def hook(name, *args)
  deprecated = {
    # none currently
  }

  if deprecated[name] and not HOOKS[name].empty? then
    warn "hook #{name} has been deprecated, use #{deprecated[name]}"
  end

  HOOKS[name].any? do |plugin|
    plugin[self, *args]
  end
end

#load_custom_extensions(config_file) ⇒ Object 



191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/autotest.rb', line 191

def load_custom_extensions(config_file)
  configs = ['./.autotest']
  if config_file
    configs << File.expand_path(config_file)
  else
    configs << File.expand_path('~/.autotest')
  end
  configs.each do |f|
    load f if File.exist? f
  end
end

#make_test_cmd(files_to_test) ⇒ Object

Generate the commands to test the supplied files



434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
 
# File 'lib/autotest.rb', line 434

def make_test_cmd files_to_test
  cmds = []
  full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
  base_cmd = "#{ruby} -I#{libs} -rubygems"

  unless full.empty? then
    classes = full.map {|k,v| k}.flatten.uniq
    classes.unshift testlib
    cmds << "#{base_cmd} -e \"[#{classes.map{|klass| "'#{klass}'"}.join(', ')}].each { |f| require f }\" | #{unit_diff}"
  end

  partial.each do |klass, methods|
    regexp = Regexp.union(*methods).source
    cmds << "#{base_cmd} #{klass} -n \"/^(#{regexp})$/\" | #{unit_diff}"
  end

  return cmds.join("#{SEP} ")
end

#new_hash_of_arrays ⇒ Object 



453
454
455
 
# File 'lib/autotest.rb', line 453

def new_hash_of_arrays
  Hash.new { |h,k| h[k] = [] }
end

#options ⇒ Object 



70
 
# File 'lib/autotest.rb', line 70

def options;@@options;end

#path_to_classname(s) ⇒ Object

Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.



320
321
322
323
324
325
326
 
# File 'lib/autotest.rb', line 320

def path_to_classname(s)
  sep = File::SEPARATOR
  f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split(sep)
  f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join }
  f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}"  }
  f.join('::')
end

#remove_exception(regexp) ⇒ Object

Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



606
607
608
609
610
 
# File 'lib/autotest.rb', line 606

def remove_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.delete regexp
  nil
end

#remove_mapping(regexp) ⇒ Object

Removed a file mapping matching regexp.



572
573
574
575
576
577
 
# File 'lib/autotest.rb', line 572

def remove_mapping regexp
  @test_mappings.delete_if do |k,v|
    k == regexp
  end
  nil
end

#reorder(files_to_test) ⇒ Object 



457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
 
# File 'lib/autotest.rb', line 457

def reorder files_to_test
  case self.order
  when :alpha then
    files_to_test.sort_by { |k,v| k }
  when :reverse then
    files_to_test.sort_by { |k,v| k }.reverse
  when :random then
    max = files_to_test.size
    files_to_test.sort_by { |k,v| rand(max) }
  when :natural then
    (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
  else
    raise "unknown order type: #{self.order.inspect}"
  end
end

#rerun_all_tests ⇒ Object

Rerun the tests from cold (reset state)



476
477
478
479
480
481
 
# File 'lib/autotest.rb', line 476

def rerun_all_tests
  reset
  run_tests

  hook :all_good if all_good
end

#reset ⇒ Object

Clear all state information about test failures and whether interrupts will kill autotest.



487
488
489
490
491
492
493
494
495
496
497
 
# File 'lib/autotest.rb', line 487

def reset
  self.files_to_test.clear
  self.find_order.clear
  self.interrupted = false
  self.known_files = nil
  self.last_mtime = T0
  self.tainted = false
  self.wants_to_quit = false

  hook :reset
end

#ruby ⇒ Object

Determine and return the path of the ruby executable.



502
503
504
505
506
507
508
509
510
 
# File 'lib/autotest.rb', line 502

def ruby
  ruby = ENV['RUBY']
  ruby ||= File.join(Config::CONFIG['bindir'],
                     Config::CONFIG['ruby_install_name'])

  ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR

  return ruby
end

#run ⇒ Object

Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.



207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
 
# File 'lib/autotest.rb', line 207

def run
  hook :initialize
  reset
  add_sigint_handler

  self.last_mtime = Time.now if options[:no_full_after_start]

  loop do
    begin # ^c handler
      get_to_green
      if self.tainted and not options[:no_full_after_failed] then
        rerun_all_tests
      else
        hook :all_good
      end
      wait_for_changes
    rescue Interrupt
      break if self.wants_to_quit
      reset
    end
  end
  hook :quit
rescue Exception => err
  hook :died, err
end

#run_tests ⇒ Object

Look for files to test then run the tests and handle the results.



246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'lib/autotest.rb', line 246

def run_tests
  hook :run_command

  new_mtime = self.find_files_to_test
  return unless new_mtime
  self.last_mtime = new_mtime

  cmd = self.make_test_cmd self.files_to_test
  return if cmd.empty?

  puts cmd unless options[:quiet]

  old_sync = $stdout.sync
  $stdout.sync = true
  self.results = []
  line = []
  begin
    open("| #{cmd}", "r") do |f|
      until f.eof? do
        c = f.getc or break
        print (c.is_a?(Fixnum) ? c.chr : c)
        line << c
        if c == ?\n then
          self.results << if RUBY_VERSION >= "1.9" then
                            line.join
                          else
                            line.pack "c*"
                          end
          line.clear
        end
      end
    end
  ensure
    $stdout.sync = old_sync
  end
  hook :ran_command
  self.results = self.results.join

  handle_results(self.results)
end

#test_files_for(filename) ⇒ Object

Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.



517
518
519
520
521
522
523
524
525
526
527
528
 
# File 'lib/autotest.rb', line 517

def test_files_for(filename)
  result = @test_mappings.find { |file_re, ignored| filename =~ file_re }

  p :test_file_for => [filename, result.first] if result and $DEBUG

  result = result.nil? ? [] : [result.last.call(filename, $~)].flatten

  output.puts "No tests matched #{filename}" if
    (options[:verbose] or $TESTING) and result.empty?

  result.sort.uniq.select { |f| known_files[f] }
end

#wait_for_changes ⇒ Object

Sleep then look for files to test, until there are some.



533
534
535
536
 
# File 'lib/autotest.rb', line 533

def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end