Class: Autotest

Inherits:
Object
  • Object
show all
Defined in:
lib/autotest.rb

Overview

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

  • Test files must be stored in test/

  • Test files names must start with test_

  • Test class names must start with Test

  • Implementation files must be stored in lib/

  • Implementation files must match up with a test file named test_.*implementation.rb

Strategy:

  1. Find all files and associate them from impl <-> test.

  2. Run all tests.

  3. Scan for failures.

  4. Detect changes in ANY (ruby?. file, rerun all failures + changed files.

  5. Until 0 defects, goto 3.

  6. When 0 defects, goto 2.

Defined Under Namespace

Modules: AutoUpdate, Once, RCov, Restart, Timestamp

Constant Summary collapse

T0 = 
Time.at 0
ALL_HOOKS = 
[ :all_good, :died, :green, :initialize, :interrupt, :quit,
:ran_command, :red, :reset, :run_command, :updated, :waiting ]
HOOKS = 
Hash.new { |h,k| h[k] = [] }
WINDOZE = 
/win32/ =~ RUBY_PLATFORM
SEP = 
WINDOZE ? '&' : ';'
@@discoveries = 
[]

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Autotest

Initialize the instance and then load the user’s .autotest file, if any.



164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
 
# File 'lib/autotest.rb', line 164

def initialize
  # these two are set directly because they're wrapped with
  # add/remove/clear accessor methods
  @exception_list = []
  @test_mappings = []

  self.completed_re = /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/
  self.extra_class_map   = {}
  self.extra_files       = []
  self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
  self.files_to_test     = new_hash_of_arrays
  self.find_order        = []
  self.known_files       = nil
  self.libs              = %w[. lib test].join(File::PATH_SEPARATOR)
  self.order             = :random
  self.output            = $stderr
  self.sleep             = 1
  self.testlib          = "test/unit"
  self.find_directories  = ['.']
  self.unit_diff         = "#{ruby} #{objectspace_opt} -s -S unit_diff -u"

  self.add_mapping(/^lib\/.*\.rb$/) do |filename, _|
    possible = File.basename(filename).gsub '_', '_?'
    files_matching %r%^test/.*#{possible}$%
  end

  self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _|
    filename
  end

  [File.expand_path('~/.autotest'), './.autotest'].each do |f|
    load f if File.exist? f
  end
end

Instance Attribute Details

#completed_re ⇒ Object

Returns the value of attribute completed_re.



142
143
144
 
# File 'lib/autotest.rb', line 142

def completed_re
  @completed_re
end

#extra_class_map ⇒ Object

Returns the value of attribute extra_class_map.



142
143
144
 
# File 'lib/autotest.rb', line 142

def extra_class_map
  @extra_class_map
end

#extra_files ⇒ Object

Returns the value of attribute extra_files.



142
143
144
 
# File 'lib/autotest.rb', line 142

def extra_files
  @extra_files
end

#failed_results_re ⇒ Object

Returns the value of attribute failed_results_re.



142
143
144
 
# File 'lib/autotest.rb', line 142

def failed_results_re
  @failed_results_re
end

#files_to_test ⇒ Object

Returns the value of attribute files_to_test.



142
143
144
 
# File 'lib/autotest.rb', line 142

def files_to_test
  @files_to_test
end

#find_directories ⇒ Object

Returns the value of attribute find_directories.



142
143
144
 
# File 'lib/autotest.rb', line 142

def find_directories
  @find_directories
end

#find_order ⇒ Object

Returns the value of attribute find_order.



142
143
144
 
# File 'lib/autotest.rb', line 142

def find_order
  @find_order
end

#interrupted ⇒ Object

Returns the value of attribute interrupted.



142
143
144
 
# File 'lib/autotest.rb', line 142

def interrupted
  @interrupted
end

#known_files ⇒ Object

Lazy accessor for the known_files hash.



420
421
422
423
424
425
 
# File 'lib/autotest.rb', line 420

def known_files
  unless @known_files then
    @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
  end
  @known_files
end

#last_mtime ⇒ Object

Returns the value of attribute last_mtime.



142
143
144
 
# File 'lib/autotest.rb', line 142

def last_mtime
  @last_mtime
end

#libs ⇒ Object

Returns the value of attribute libs.



142
143
144
 
# File 'lib/autotest.rb', line 142

def libs
  @libs
end

#order ⇒ Object

Returns the value of attribute order.



142
143
144
 
# File 'lib/autotest.rb', line 142

def order
  @order
end

#output ⇒ Object

Returns the value of attribute output.



142
143
144
 
# File 'lib/autotest.rb', line 142

def output
  @output
end

#results ⇒ Object

Returns the value of attribute results.



142
143
144
 
# File 'lib/autotest.rb', line 142

def results
  @results
end

#sleep ⇒ Object

Returns the value of attribute sleep.



142
143
144
 
# File 'lib/autotest.rb', line 142

def sleep
  @sleep
end

#tainted ⇒ Object

Returns the value of attribute tainted.



142
143
144
 
# File 'lib/autotest.rb', line 142

def tainted
  @tainted
end

#testlib ⇒ Object

Returns the value of attribute testlib.



142
143
144
 
# File 'lib/autotest.rb', line 142

def testlib
  @testlib
end

#unit_diff ⇒ Object

Returns the value of attribute unit_diff.



142
143
144
 
# File 'lib/autotest.rb', line 142

def unit_diff
  @unit_diff
end

#wants_to_quit ⇒ Object

Returns the value of attribute wants_to_quit.



142
143
144
 
# File 'lib/autotest.rb', line 142

def wants_to_quit
  @wants_to_quit
end

Class Method Details

.add_discovery(&proc) ⇒ Object

Add a proc to the collection of discovery procs. See autodiscover.



79
80
81
 
# File 'lib/autotest.rb', line 79

def self.add_discovery &proc
  @@discoveries << proc
end

.add_hook(name, &block) ⇒ Object

Add the supplied block to the available hooks, with the given name.



668
669
670
 
# File 'lib/autotest.rb', line 668

def self.add_hook(name, &block)
  HOOKS[name] << block
end

.autodiscover ⇒ Object

Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb”. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.

Process:

  1. All autotest/discover.rb files loaded.

  2. Those procs determine your styles (eg [“rails”, “rspec”]).

  3. Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).

  4. Invoke run method on appropriate class (eg Autotest::RailsRspec.run).

Example autotest/discover.rb:

Autotest.add_discovery do
  "rails" if File.exist? 'config/environment.rb'
end



108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/autotest.rb', line 108

def self.autodiscover
  style = []

  paths = $:.dup
  paths.push 'lib'
  paths.push(*Dir["vendor/plugins/*/lib"])

  begin
    require 'rubygems'
    paths.push(*Gem.latest_load_paths)
  rescue LoadError => e
    # do nothing
  end

  paths.each do |d|
    next unless File.directory? d
    f = File.join(d, 'autotest', 'discover.rb')
    if File.exist? f then
      $: << d unless $:.include? d
      load f
    end
  end

  @@discoveries.map { |proc| proc.call }.flatten.compact.sort.uniq
end

.run ⇒ Object

Initialize and run the system.



137
138
139
 
# File 'lib/autotest.rb', line 137

def self.run
  new.run
end

Instance Method Details

#add_exception(regexp) ⇒ Object

Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



599
600
601
602
603
 
# File 'lib/autotest.rb', line 599

def add_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list << regexp
  nil
end

#add_mapping(regexp, prepend = false, &proc) ⇒ Object

Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.

For example, if test_helper.rb is modified, rerun all tests:

at.add_mapping(/test_helper.rb/) do |f, _|
  at.files_matching(/^test.*rb$/)
end



563
564
565
566
567
568
569
570
 
# File 'lib/autotest.rb', line 563

def add_mapping(regexp, prepend = false, &proc)
  if prepend then
    @test_mappings.unshift [regexp, proc]
  else
    @test_mappings.push [regexp, proc]
  end
  nil
end

#add_sigint_handler ⇒ Object

Installs a sigint handler.



289
290
291
292
293
294
295
296
297
298
299
300
301
302
 
# File 'lib/autotest.rb', line 289

def add_sigint_handler
  trap 'INT' do
    if self.interrupted then
      self.wants_to_quit = true
    else
      unless hook :interrupt then
        puts "Interrupt a second time to quit"
        self.interrupted = true
        Kernel.sleep 1.5
      end
      raise Interrupt, nil # let the run loop catch it
    end
  end
end

#all_good ⇒ Object

If there are no files left to test (because they’ve all passed), then all is good.



308
309
310
 
# File 'lib/autotest.rb', line 308

def all_good
  files_to_test.empty?
end

#clear_exceptions ⇒ Object

Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.



619
620
621
622
623
 
# File 'lib/autotest.rb', line 619

def clear_exceptions
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.clear
  nil
end

#clear_mappings ⇒ Object

Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.



587
588
589
590
 
# File 'lib/autotest.rb', line 587

def clear_mappings
  @test_mappings.clear
  nil
end

#consolidate_failures(failed) ⇒ Object

Returns a hash mapping a file name to the known failures for that file.



328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
 
# File 'lib/autotest.rb', line 328

def consolidate_failures(failed)
  filters = new_hash_of_arrays

  class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly
                     [path_to_classname(f), f]
                   }.flatten]
  class_map.merge!(self.extra_class_map)

  failed.each do |method, klass|
    if class_map.has_key? klass then
      filters[class_map[klass]] << method
    else
      output.puts "Unable to map class #{klass} to a file"
    end
  end

  return filters
end

#exceptions ⇒ Object

Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.



630
631
632
633
634
635
636
637
638
639
640
 
# File 'lib/autotest.rb', line 630

def exceptions
  unless defined? @exceptions then
    if @exception_list.empty? then
      @exceptions = nil
    else
      @exceptions = Regexp.union(*@exception_list)
    end
  end

  @exceptions
end

#files_matching(regexp) ⇒ Object

Returns all known files in the codebase matching regexp.



547
548
549
 
# File 'lib/autotest.rb', line 547

def files_matching regexp
  self.find_order.select { |k| k =~ regexp }
end

#find_files ⇒ Object

Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.



352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
 
# File 'lib/autotest.rb', line 352

def find_files
  result = {}
  targets = self.find_directories + self.extra_files
  self.find_order.clear

  targets.each do |target|
    order = []
    Find.find(target) do |f|
      Find.prune if f =~ self.exceptions

      next if test ?d, f
      next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files
      next if f =~ /\/\.?#/            # Emacs autosave/cvs merge files

      filename = f.sub(/^\.\//, '')

      result[filename] = File.stat(filename).mtime rescue next
      order << filename
    end
    self.find_order.push(*order.sort)
  end

  return result
end

#find_files_to_test(files = find_files) ⇒ Object

Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns true if any file is newer than the previously recorded most recent file.



383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
 
# File 'lib/autotest.rb', line 383

def find_files_to_test(files=find_files)
  updated = files.select { |filename, mtime| self.last_mtime < mtime }

  p updated if $v unless updated.empty? || self.last_mtime.to_i == 0

  hook :updated, updated unless updated.empty? || self.last_mtime.to_i == 0

  updated.map { |f,m| test_files_for(f) }.flatten.uniq.each do |filename|
    self.files_to_test[filename] # creates key with default value
  end

  if updated.empty? then
    nil
  else
    files.values.max
  end
end

#get_to_green ⇒ Object

Keep running the tests after a change, until all pass.



232
233
234
235
236
237
 
# File 'lib/autotest.rb', line 232

def get_to_green
  begin
    run_tests
    wait_for_changes unless all_good
  end until all_good
end

#handle_results(results) ⇒ Object

Check results for failures, set the “bar” to red or green, and if there are failures record this.



405
406
407
408
409
410
411
412
413
414
415
 
# File 'lib/autotest.rb', line 405

def handle_results(results)
  failed = results.scan(self.failed_results_re)
  completed = results =~ self.completed_re

  self.files_to_test = consolidate_failures failed if completed

  color = completed && self.files_to_test.empty? ? :green : :red
  hook color unless $TESTING

  self.tainted = true unless self.files_to_test.empty?
end

#hook(name, *args) ⇒ Object

Call the event hook named name, executing all registered hooks until one returns true. Returns false if no hook handled the event.



650
651
652
653
654
655
656
657
658
659
660
661
662
 
# File 'lib/autotest.rb', line 650

def hook(name, *args)
  deprecated = {
    # none currently
  }

  if deprecated[name] and not HOOKS[name].empty? then
    warn "hook #{name} has been deprecated, use #{deprecated[name]}"
  end

  HOOKS[name].any? do |plugin|
    plugin[self, *args]
  end
end

#make_test_cmd(files_to_test) ⇒ Object

Generate the commands to test the supplied files



430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
 
# File 'lib/autotest.rb', line 430

def make_test_cmd files_to_test
  cmds = []
  full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
  base_cmd = "#{ruby} #{objectspace_opt} -I#{libs} -rubygems"

  unless full.empty? then
    classes = full.map {|k,v| k}.flatten.uniq
    classes.unshift testlib
    cmds << "#{base_cmd} -e \"%w[#{classes.join(' ')}].each { |f| require f }\" | #{unit_diff}"
  end

  partial.each do |klass, methods|
    regexp = Regexp.union(*methods).source
    cmds << "#{base_cmd} #{klass} -n \"/^(#{regexp})$/\" | #{unit_diff}"
  end

  return cmds.join("#{SEP} ")
end

#new_hash_of_arrays ⇒ Object 



449
450
451
 
# File 'lib/autotest.rb', line 449

def new_hash_of_arrays
  Hash.new { |h,k| h[k] = [] }
end

#objectspace_opt ⇒ Object

Enable ObjectSpace for JRuby



511
512
513
 
# File 'lib/autotest.rb', line 511

def objectspace_opt
  RUBY_PLATFORM =~ /java/ ? '-X+O' : ''
end

#path_to_classname(s) ⇒ Object

Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.



316
317
318
319
320
321
322
 
# File 'lib/autotest.rb', line 316

def path_to_classname(s)
  sep = File::SEPARATOR
  f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split(sep)
  f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join }
  f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}"  }
  f.join('::')
end

#remove_exception(regexp) ⇒ Object

Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



609
610
611
612
613
 
# File 'lib/autotest.rb', line 609

def remove_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.delete regexp
  nil
end

#remove_mapping(regexp) ⇒ Object

Removed a file mapping matching regexp.



575
576
577
578
579
580
 
# File 'lib/autotest.rb', line 575

def remove_mapping regexp
  @test_mappings.delete_if do |k,v|
    k == regexp
  end
  nil
end

#reorder(files_to_test) ⇒ Object 



453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
 
# File 'lib/autotest.rb', line 453

def reorder files_to_test
  case self.order
  when :alpha then
    files_to_test.sort_by { |k,v| k }
  when :reverse then
    files_to_test.sort_by { |k,v| k }.reverse
  when :random then
    max = files_to_test.size
    files_to_test.sort_by { |k,v| rand(max) }
  when :natural then
    (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
  else
    raise "unknown order type: #{self.order.inspect}"
  end
end

#rerun_all_tests ⇒ Object

Rerun the tests from cold (reset state)



472
473
474
475
476
477
 
# File 'lib/autotest.rb', line 472

def rerun_all_tests
  reset
  run_tests

  hook :all_good if all_good
end

#reset ⇒ Object

Clear all state information about test failures and whether interrupts will kill autotest.



483
484
485
486
487
488
489
490
491
492
493
 
# File 'lib/autotest.rb', line 483

def reset
  self.files_to_test.clear
  self.find_order.clear
  self.interrupted = false
  self.known_files = nil
  self.last_mtime = T0
  self.tainted = false
  self.wants_to_quit = false

  hook :reset
end

#ruby ⇒ Object

Determine and return the path of the ruby executable.



498
499
500
501
502
503
504
505
506
 
# File 'lib/autotest.rb', line 498

def ruby
  ruby = ENV['RUBY']
  ruby ||= File.join(Config::CONFIG['bindir'],
                     Config::CONFIG['ruby_install_name'])

  ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR

  return ruby
end

#run ⇒ Object

Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.



203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
 
# File 'lib/autotest.rb', line 203

def run
  hook :initialize
  reset
  add_sigint_handler

  self.last_mtime = Time.now if $f

  loop do # ^c handler
    begin
      get_to_green
      if self.tainted then
        rerun_all_tests
      else
        hook :all_good
      end
      wait_for_changes
    rescue Interrupt
      break if self.wants_to_quit
      reset
    end
  end
  hook :quit
rescue Exception
  hook :died
end

#run_tests ⇒ Object

Look for files to test then run the tests and handle the results.



242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
 
# File 'lib/autotest.rb', line 242

def run_tests
  hook :run_command

  new_mtime = self.find_files_to_test
  return unless new_mtime
  self.last_mtime = new_mtime

  cmd = self.make_test_cmd self.files_to_test
  return if cmd.empty?

  puts cmd unless $q

  old_sync = $stdout.sync
  $stdout.sync = true
  self.results = []
  line = []
  begin
    open("| #{cmd}", "r") do |f|
      until f.eof? do
        c = f.getc
        putc c
        line << c
        if c == ?\n then
          self.results << if RUBY_VERSION >= "1.9" then
                            line.join
                          else
                            line.pack "c*"
                          end
          line.clear
        end
      end
    end
  ensure
    $stdout.sync = old_sync
  end
  hook :ran_command
  self.results = self.results.join

  handle_results(self.results)
end

#test_files_for(filename) ⇒ Object

Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.



520
521
522
523
524
525
526
527
528
529
530
531
 
# File 'lib/autotest.rb', line 520

def test_files_for(filename)
  result = @test_mappings.find { |file_re, ignored| filename =~ file_re }

  p :test_file_for => [filename, result.first] if result and $DEBUG

  result = result.nil? ? [] : Array(result.last.call(filename, $~))

  output.puts "No tests matched #{filename}" if
    ($v or $TESTING) and result.empty?

  result.sort.uniq.select { |f| known_files[f] }
end

#wait_for_changes ⇒ Object

Sleep then look for files to test, until there are some.



536
537
538
539
 
# File 'lib/autotest.rb', line 536

def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end