Class: Autotest

Inherits:
Object
  • Object
show all
Defined in:
lib/autotest.rb

Overview

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

  • Test files must be stored in test/

  • Test files names must start with test_

  • Test class names must start with Test

  • Implementation files must be stored in lib/

  • Implementation files must match up with a test file named test_.*implementation.rb

Strategy:

  1. Find all files and associate them from impl <-> test.

  2. Run all tests.

  3. Scan for failures.

  4. Detect changes in ANY (ruby?. file, rerun all failures + changed files.

  5. Until 0 defects, goto 3.

  6. When 0 defects, goto 2.

Defined Under Namespace

Modules: AutoUpdate, Once, RCov, Restart, Timestamp

Constant Summary collapse

RUBY19 = 
defined? Encoding
T0 = 
Time.at 0
ALL_HOOKS = 
[ :all_good, :died, :green, :initialize, :interrupt, :quit,
:ran_command, :red, :reset, :run_command, :updated, :waiting ]
HOOKS = 
Hash.new { |h,k| h[k] = [] }
WINDOZE = 
/win32/ =~ RUBY_PLATFORM
SEP = 
WINDOZE ? '&' : ';'
@@discoveries = 
[]

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Autotest

Initialize the instance and then load the user’s .autotest file, if any.



152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
 
# File 'lib/autotest.rb', line 152

def initialize
  # these two are set directly because they're wrapped with
  # add/remove/clear accessor methods
  @exception_list = []
  @test_mappings = []

  self.completed_re = /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/
  self.extra_class_map   = {}
  self.extra_files       = []
  self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
  self.files_to_test     = new_hash_of_arrays
  self.find_order        = []
  self.known_files       = nil
  self.libs              = %w[. lib test].join(File::PATH_SEPARATOR)
  self.order             = :random
  self.output            = $stderr
  self.sleep             = 1
  self.testlib           = "test/unit"
  self.find_directories  = ['.']
  self.unit_diff         = "unit_diff -u"

  self.add_mapping(/^lib\/.*\.rb$/) do |filename, _|
    possible = File.basename(filename).gsub '_', '_?'
    files_matching %r%^test/.*#{possible}$%
  end

  self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _|
    filename
  end

  [File.expand_path('~/.autotest'), './.autotest'].each do |f|
    load f if File.exist? f
  end
end

Instance Attribute Details

#completed_re ⇒ Object

Returns the value of attribute completed_re.



128
129
130
 
# File 'lib/autotest.rb', line 128

def completed_re
  @completed_re
end

#extra_class_map ⇒ Object

Returns the value of attribute extra_class_map.



128
129
130
 
# File 'lib/autotest.rb', line 128

def extra_class_map
  @extra_class_map
end

#extra_files ⇒ Object

Returns the value of attribute extra_files.



128
129
130
 
# File 'lib/autotest.rb', line 128

def extra_files
  @extra_files
end

#failed_results_re ⇒ Object

Returns the value of attribute failed_results_re.



128
129
130
 
# File 'lib/autotest.rb', line 128

def failed_results_re
  @failed_results_re
end

#files_to_test ⇒ Object

Returns the value of attribute files_to_test.



128
129
130
 
# File 'lib/autotest.rb', line 128

def files_to_test
  @files_to_test
end

#find_directories ⇒ Object

Returns the value of attribute find_directories.



128
129
130
 
# File 'lib/autotest.rb', line 128

def find_directories
  @find_directories
end

#find_order ⇒ Object

Returns the value of attribute find_order.



128
129
130
 
# File 'lib/autotest.rb', line 128

def find_order
  @find_order
end

#interrupted ⇒ Object

Returns the value of attribute interrupted.



128
129
130
 
# File 'lib/autotest.rb', line 128

def interrupted
  @interrupted
end

#known_files ⇒ Object

Lazy accessor for the known_files hash.



413
414
415
416
417
418
 
# File 'lib/autotest.rb', line 413

def known_files
  unless @known_files then
    @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
  end
  @known_files
end

#last_mtime ⇒ Object

Returns the value of attribute last_mtime.



128
129
130
 
# File 'lib/autotest.rb', line 128

def last_mtime
  @last_mtime
end

#libs ⇒ Object

Returns the value of attribute libs.



128
129
130
 
# File 'lib/autotest.rb', line 128

def libs
  @libs
end

#order ⇒ Object

Returns the value of attribute order.



128
129
130
 
# File 'lib/autotest.rb', line 128

def order
  @order
end

#output ⇒ Object

Returns the value of attribute output.



128
129
130
 
# File 'lib/autotest.rb', line 128

def output
  @output
end

#results ⇒ Object

Returns the value of attribute results.



128
129
130
 
# File 'lib/autotest.rb', line 128

def results
  @results
end

#sleep ⇒ Object

Returns the value of attribute sleep.



128
129
130
 
# File 'lib/autotest.rb', line 128

def sleep
  @sleep
end

#tainted ⇒ Object Also known as: tainted?

Returns the value of attribute tainted.



128
129
130
 
# File 'lib/autotest.rb', line 128

def tainted
  @tainted
end

#testlib ⇒ Object

Returns the value of attribute testlib.



128
129
130
 
# File 'lib/autotest.rb', line 128

def testlib
  @testlib
end

#unit_diff ⇒ Object

Returns the value of attribute unit_diff.



128
129
130
 
# File 'lib/autotest.rb', line 128

def unit_diff
  @unit_diff
end

#wants_to_quit ⇒ Object

Returns the value of attribute wants_to_quit.



128
129
130
 
# File 'lib/autotest.rb', line 128

def wants_to_quit
  @wants_to_quit
end

Class Method Details

.add_discovery(&proc) ⇒ Object

Add a proc to the collection of discovery procs. See autodiscover.



81
82
83
 
# File 'lib/autotest.rb', line 81

def self.add_discovery &proc
  @@discoveries << proc
end

.add_hook(name, &block) ⇒ Object

Add the supplied block to the available hooks, with the given name.



655
656
657
 
# File 'lib/autotest.rb', line 655

def self.add_hook(name, &block)
  HOOKS[name] << block
end

.autodiscover ⇒ Object

Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb”. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.

Process:

  1. All autotest/discover.rb files loaded.

  2. Those procs determine your styles (eg [“rails”, “rspec”]).

  3. Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).

  4. Invoke run method on appropriate class (eg Autotest::RailsRspec.run).

Example autotest/discover.rb:

Autotest.add_discovery do
  "rails" if File.exist? 'config/environment.rb'
end



110
111
112
113
114
115
116
117
118
 
# File 'lib/autotest.rb', line 110

def self.autodiscover
  require 'rubygems'

  Gem.find_files("autotest/discover").each do |f|
    load f
  end

  @@discoveries.map { |proc| proc.call }.flatten.compact.sort.uniq
end

.run ⇒ Object

Initialize and run the system.



123
124
125
 
# File 'lib/autotest.rb', line 123

def self.run
  new.run
end

Instance Method Details

#add_exception(regexp) ⇒ Object

Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



585
586
587
588
589
590
 
# File 'lib/autotest.rb', line 585

def add_exception regexp
  raise "exceptions already compiled" if defined? @exceptions

  @exception_list << regexp
  nil
end

#add_mapping(regexp, prepend = false, &proc) ⇒ Object

Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.

For example, if test_helper.rb is modified, rerun all tests:

at.add_mapping(/test_helper.rb/) do |f, _|
  at.files_matching(/^test.*rb$/)
end



549
550
551
552
553
554
555
556
 
# File 'lib/autotest.rb', line 549

def add_mapping(regexp, prepend = false, &proc)
  if prepend then
    @test_mappings.unshift [regexp, proc]
  else
    @test_mappings.push [regexp, proc]
  end
  nil
end

#add_sigint_handler ⇒ Object

Installs a sigint handler.



281
282
283
284
285
286
287
288
289
290
291
292
293
294
 
# File 'lib/autotest.rb', line 281

def add_sigint_handler
  trap 'INT' do
    if self.interrupted then
      self.wants_to_quit = true
    else
      unless hook :interrupt then
        puts "Interrupt a second time to quit"
        self.interrupted = true
        Kernel.sleep 1.5
      end
      raise Interrupt, nil # let the run loop catch it
    end
  end
end

#all_good ⇒ Object

If there are no files left to test (because they’ve all passed), then all is good.



300
301
302
 
# File 'lib/autotest.rb', line 300

def all_good
  files_to_test.empty?
end

#clear_exceptions ⇒ Object

Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.



606
607
608
609
610
 
# File 'lib/autotest.rb', line 606

def clear_exceptions
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.clear
  nil
end

#clear_mappings ⇒ Object

Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.



573
574
575
576
 
# File 'lib/autotest.rb', line 573

def clear_mappings
  @test_mappings.clear
  nil
end

#consolidate_failures(failed) ⇒ Object

Returns a hash mapping a file name to the known failures for that file.



321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
 
# File 'lib/autotest.rb', line 321

def consolidate_failures(failed)
  filters = new_hash_of_arrays

  class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly
                     [path_to_classname(f), f]
                   }.flatten]
  class_map.merge!(self.extra_class_map)

  failed.each do |method, klass|
    if class_map.has_key? klass then
      filters[class_map[klass]] << method
    else
      output.puts "Unable to map class #{klass} to a file"
    end
  end

  return filters
end

#exceptions ⇒ Object

Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.



617
618
619
620
621
622
623
624
625
626
627
 
# File 'lib/autotest.rb', line 617

def exceptions
  unless defined? @exceptions then
    if @exception_list.empty? then
      @exceptions = nil
    else
      @exceptions = Regexp.union(*@exception_list)
    end
  end

  @exceptions
end

#files_matching(regexp) ⇒ Object

Returns all known files in the codebase matching regexp.



533
534
535
 
# File 'lib/autotest.rb', line 533

def files_matching regexp
  self.find_order.select { |k| k =~ regexp }
end

#find_files ⇒ Object

Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.



345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
 
# File 'lib/autotest.rb', line 345

def find_files
  result = {}
  targets = self.find_directories + self.extra_files
  self.find_order.clear

  targets.each do |target|
    order = []
    Find.find(target) do |f|
      Find.prune if f =~ self.exceptions

      next if test ?d, f
      next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files
      next if f =~ /\/\.?#/            # Emacs autosave/cvs merge files

      filename = f.sub(/^\.\//, '')

      result[filename] = File.stat(filename).mtime rescue next
      order << filename
    end
    self.find_order.push(*order.sort)
  end

  return result
end

#find_files_to_test(files = find_files) ⇒ Object

Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns true if any file is newer than the previously recorded most recent file.



376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
 
# File 'lib/autotest.rb', line 376

def find_files_to_test files = find_files
  updated = files.select { |filename, mtime| self.last_mtime < mtime }

  p updated if $v unless updated.empty? || self.last_mtime.to_i == 0

  hook :updated, updated unless updated.empty? || self.last_mtime.to_i == 0

  updated.map { |f,m| test_files_for(f) }.flatten.uniq.each do |filename|
    self.files_to_test[filename] # creates key with default value
  end

  if updated.empty? then
    nil
  else
    files.values.max
  end
end

#get_to_green ⇒ Object

Keep running the tests after a change, until all pass.



220
221
222
223
224
225
 
# File 'lib/autotest.rb', line 220

def get_to_green
  begin
    run_tests
    wait_for_changes unless all_good
  end until all_good
end

#handle_results(results) ⇒ Object

Check results for failures, set the “bar” to red or green, and if there are failures record this.



398
399
400
401
402
403
404
405
406
407
408
 
# File 'lib/autotest.rb', line 398

def handle_results(results)
  failed = results.scan(self.failed_results_re)
  completed = results =~ self.completed_re

  self.files_to_test = consolidate_failures failed if completed

  color = completed && self.files_to_test.empty? ? :green : :red
  hook color unless $TESTING

  self.tainted = true unless self.files_to_test.empty?
end

#hook(name, *args) ⇒ Object

Call the event hook named name, executing all registered hooks until one returns true. Returns false if no hook handled the event.



637
638
639
640
641
642
643
644
645
646
647
648
649
 
# File 'lib/autotest.rb', line 637

def hook(name, *args)
  deprecated = {
    # none currently
  }

  if deprecated[name] and not HOOKS[name].empty? then
    warn "hook #{name} has been deprecated, use #{deprecated[name]}"
  end

  HOOKS[name].any? do |plugin|
    plugin[self, *args]
  end
end

#make_test_cmd(files_to_test) ⇒ Object

Generate the commands to test the supplied files



423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
 
# File 'lib/autotest.rb', line 423

def make_test_cmd files_to_test
  cmds = []
  full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
  base_cmd = "#{ruby} -I#{libs} -rubygems"

  unless full.empty? then
    classes = full.map {|k,v| k}.flatten.uniq
    classes.unshift testlib
    cmds << "#{base_cmd} -e \"%w[#{classes.join(' ')}].each { |f| require f }\" | #{unit_diff}"
  end

  partial.each do |klass, methods|
    regexp = Regexp.union(*methods).source
    cmds << "#{base_cmd} #{klass} -n \"/^(#{regexp})$/\" | #{unit_diff}"
  end

  return cmds.join("#{SEP} ")
end

#new_hash_of_arrays ⇒ Object 



442
443
444
 
# File 'lib/autotest.rb', line 442

def new_hash_of_arrays
  Hash.new { |h,k| h[k] = [] }
end

#path_to_classname(s) ⇒ Object

Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.



308
309
310
311
312
313
314
315
 
# File 'lib/autotest.rb', line 308

def path_to_classname(s)
  sep = File::SEPARATOR
  f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split(sep)
  f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join }
  f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}"  }

  f.join('::')
end

#remove_exception(regexp) ⇒ Object

Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.



596
597
598
599
600
 
# File 'lib/autotest.rb', line 596

def remove_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.delete regexp
  nil
end

#remove_mapping(regexp) ⇒ Object

Removed a file mapping matching regexp.



561
562
563
564
565
566
 
# File 'lib/autotest.rb', line 561

def remove_mapping regexp
  @test_mappings.delete_if do |k,v|
    k == regexp
  end
  nil
end

#reorder(files_to_test) ⇒ Object 



446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
 
# File 'lib/autotest.rb', line 446

def reorder files_to_test
  case self.order
  when :alpha then
    files_to_test.sort_by { |k,v| k }
  when :reverse then
    files_to_test.sort_by { |k,v| k }.reverse
  when :random then
    max = files_to_test.size
    files_to_test.sort_by { |k,v| rand(max) }
  when :natural then
    (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
  else
    raise "unknown order type: #{self.order.inspect}"
  end
end

#rerun_all_tests ⇒ Object

Rerun the tests from cold (reset state)



465
466
467
468
469
470
 
# File 'lib/autotest.rb', line 465

def rerun_all_tests
  reset
  run_tests

  hook :all_good if all_good
end

#reset ⇒ Object

Clear all state information about test failures and whether interrupts will kill autotest.



476
477
478
479
480
481
482
483
484
485
486
 
# File 'lib/autotest.rb', line 476

def reset
  self.files_to_test.clear
  self.find_order.clear
  self.interrupted = false
  self.known_files = nil
  self.last_mtime = T0
  self.tainted = false
  self.wants_to_quit = false

  hook :reset
end

#ruby ⇒ Object

Determine and return the path of the ruby executable.



491
492
493
494
495
496
497
498
499
 
# File 'lib/autotest.rb', line 491

def ruby
  ruby = ENV['RUBY']
  ruby ||= File.join(Config::CONFIG['bindir'],
                     Config::CONFIG['ruby_install_name'])

  ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR

  return ruby
end

#run ⇒ Object

Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
 
# File 'lib/autotest.rb', line 191

def run
  hook :initialize
  reset
  add_sigint_handler

  self.last_mtime = Time.now if $f

  loop do # ^c handler
    begin
      get_to_green
      if tainted? then
        rerun_all_tests
      else
        hook :all_good
      end
      wait_for_changes
    rescue Interrupt
      break if wants_to_quit
      reset
    end
  end
  hook :quit
rescue Exception => err
  hook :died, err
end

#run_tests ⇒ Object

Look for files to test then run the tests and handle the results.



230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
 
# File 'lib/autotest.rb', line 230

def run_tests
  hook :run_command

  new_mtime = self.find_files_to_test
  return unless new_mtime
  self.last_mtime = new_mtime

  cmd = self.make_test_cmd self.files_to_test
  return if cmd.empty?

  puts cmd unless $q

  old_sync = $stdout.sync
  $stdout.sync = true
  self.results = []
  line = []
  begin
    open("| #{cmd}", "r") do |f|
      until f.eof? do
        c = f.getc or break
        if RUBY19 then
          print c
        else
          putc c
        end
        line << c
        if c == ?\n then
          self.results << if RUBY19 then
                            line.join
                          else
                            line.pack "c*"
                          end
          line.clear
        end
      end
    end
  ensure
    $stdout.sync = old_sync
  end
  hook :ran_command
  self.results = self.results.join

  handle_results(self.results)
end

#test_files_for(filename) ⇒ Object

Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.



506
507
508
509
510
511
512
513
514
515
516
517
 
# File 'lib/autotest.rb', line 506

def test_files_for(filename)
  result = @test_mappings.find { |file_re, ignored| filename =~ file_re }

  p :test_file_for => [filename, result.first] if result and $DEBUG

  result = result.nil? ? [] : [result.last.call(filename, $~)].flatten

  output.puts "No tests matched #{filename}" if
    ($v or $TESTING) and result.empty?

  result.sort.uniq.select { |f| known_files[f] }
end

#wait_for_changes ⇒ Object

Sleep then look for files to test, until there are some.



522
523
524
525
 
# File 'lib/autotest.rb', line 522

def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end