Class: BareTest::Run

Inherits:
Object
  • Object
show all
Defined in:
lib/baretest/run.rb,
lib/baretest/run/cli.rb,
lib/baretest/run/tap.rb,
lib/baretest/run/xml.rb,
lib/baretest/run/none.rb,
lib/baretest/run/spec.rb,
lib/baretest/run/minimal.rb

Overview

Run is the environment in which the suites and assertions are executed. Prior to the execution, the Run instance extends itself with the formatter that should be used and other runner related toolsets. Your formatter can override:

:run_all

Invoked once, before the first run_suite is ran. No arguments.

:run_suite

Invoked per suite. Takes the suite to run as argument.

:run_test_variants

Invoked per assertion. Takes the assertion to execute as argument.

:run_test

Invoked per setup variation of each assertion. Takes the assertion to execute and the setup blocks to use as arguments.

Don’t forget to call super within your overrides, or the tests won’t be executed.

Defined Under Namespace

Modules: CLI, Minimal, None, Spec, TAP, XML

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(suite, opts = nil) ⇒ Run

Run the passed suite. Calls run_all with the toplevel suite as argument and a block that calls run_suite with the yielded argument (which should be the toplevel suite). Options accepted:

  • :extenders: An Array of Modules, will be used as argument to self.extend, useful e.g. for mock integration

  • :format: A string with the basename (without suffix) of the formatter to use - or a Module

  • :interactive: true/false, will switch this Test::Run instance into IRB mode, where an error will cause an irb session to be started in the context of a clean copy of the assertion with all setup callbacks invoked

The order of extensions is:

  • :extender

  • :format (extends with the formatter module)

  • :interactive (extends with IRBMode)



61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
 
# File 'lib/baretest/run.rb', line 61

def initialize(suite, opts=nil)
  @suite           = suite
  @inits           = []
  @options         = opts || {}
  @count           = @options[:count] || Hash.new(0)
  @provided        = [] # Array's set operations are the fastest
  @include_tags    = @options[:include_tags]   # nil is ok here
  @exclude_tags    = @options[:exclude_tags]   # nil is ok here
  include_states   = @options[:include_states] # nil is ok here
  exclude_states   = @options[:exclude_states] # nil is ok here
  @states          = [nil, *BareTest::StatusOrder]
  @skipped         = {}
  @last_run_states = {}
  @persistence     = @options[:persistence]

  if (include_states || exclude_states) && !((include_states && include_states.empty?) && (exclude_states && exclude_states.empty?)) then
    [include_states, exclude_states].compact.each do |states|
      states << nil if states.include?(:new)
      states.push(:error, :skipped, :pending) if states.include?(:failure)
      states.delete(:new)
      if states.include?(:skipped) then
        states.delete(:skipped)
        states.push(:pending, :manually_skipped, :dependency_missing, :library_missing, :component_missing)
      end
      states.uniq!
    end
    @states = (include_states || @states) - (exclude_states || [])
  end

  (BareTest.extender+Array(@options[:extender])).each do |extender|
    extend(extender)
  end

  # Extend with the output formatter
  format = @options[:format]
  if format.is_a?(String) then
    require "baretest/run/#{format}"
    extend(BareTest.format["baretest/run/#{format}"])
  elsif format.is_a?(Module) then
    extend(format)
  end

  # Extend with irb dropout code
  extend(BareTest::IRBMode) if @options[:interactive]

  # Initialize extenders
  @inits.each { |init| instance_eval(&init) }
end

Instance Attribute Details

#count ⇒ Object (readonly)

Some statistics, standard count keys are:

:test

the number of tests executed until now

:suite

the number of suites executed until now

<status>

the number of tests with that status (see BareTest::StatusOrder for a list of states)



42
43
44
 
# File 'lib/baretest/run.rb', line 42

def count
  @count
end

#inits ⇒ Object (readonly)

The initialisation blocks of extenders



35
36
37
 
# File 'lib/baretest/run.rb', line 35

def inits
  @inits
end

#suite ⇒ Object (readonly)

The toplevel suite.



32
33
34
 
# File 'lib/baretest/run.rb', line 32

def suite
  @suite
end

Instance Method Details

#global_status ⇒ Object

Status over all tests ran up to now Can be :error, :failure, :incomplete or :success The algorithm is a simple fall through: if any test errored, then global_status is :error, if not, then if any test failed, global_status is :failure, if not, then if any test was pending or skipped, global_status is :incomplete, if not, then global_status is success



237
238
239
240
241
242
 
# File 'lib/baretest/run.rb', line 237

def global_status
  status_counts         = @count.values_at(*BareTest::StatusOrder)
  most_important_status = BareTest::StatusOrder.zip(status_counts) { |status, count|
    break status if count > 0
  } || :success
end

#init(&block) ⇒ Object

Hook initializers for extenders. Blocks passed to init will be instance_eval’d at the end of initialize. Example usage:

module ExtenderForRun
  def self.extended(run_obj)
     run_obj.init do
       # do some initialization stuff for this module
     end
  end
end



120
121
122
 
# File 'lib/baretest/run.rb', line 120

def init(&block)
  @inits << block
end

#interpolated_description(assertion, setups) ⇒ Object

Get an assertions’ interpolated description for a given Array of Setup instances. See Assertion#interpolated_description



247
248
249
250
251
252
 
# File 'lib/baretest/run.rb', line 247

def interpolated_description(assertion, setups)
  setups = setups ? setups.select { |s| s.component } : []
  substitutes = {}
  setups.each do |setup| substitutes[setup.component] = setup.substitute end
  assertion.interpolated_description(substitutes)
end

#run_all ⇒ Object

Formatter callback. Invoked once at the beginning. Gets the toplevel suite as single argument.



127
128
129
130
131
132
 
# File 'lib/baretest/run.rb', line 127

def run_all
  @last_run_states = @persistence ? @persistence.read('final_states', {}) : {}
  @skipped         = {}
  run_suite(@suite)
  @persistence.store('final_states', @last_run_states) if @persistence
end

#run_suite(suite) ⇒ Object

Formatter callback. Invoked once for every suite. Gets the suite to run as single argument. Runs all assertions and nested suites.



138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'lib/baretest/run.rb', line 138

def run_suite(suite)
  missing_tags     = @include_tags && @include_tags - suite.tags
  superfluous_tags = @exclude_tags && suite.tags & @exclude_tags
  ignored          = (missing_tags && !missing_tags.empty?) || (superfluous_tags && !superfluous_tags.empty?)

  unless ignored then
    unmet_dependencies  = (suite.depends_on-@provided)
    manually_skipped    = suite.skipped?
    recursively_skipped = !unmet_dependencies.empty? || manually_skipped
    skipped             = @skipped[suite] || recursively_skipped

    if recursively_skipped then
      skip_recursively(suite, "Ancestor was skipped")
    elsif skipped then
      skip_suite(suite, "Container was skipped")
    end
  end

  if ignored then
    states = []
  else
    states = suite.assertions.map do |test|
      run_test_variants(test)
    end
  end
  states.concat(suite.suites.map { |(description, subsuite)|
    run_suite(subsuite)
  })
  @count[:suite] += 1

  # || in case the suite contains no tests or suites
  final_status = BareTest.most_important_status(states) || :pending

  @provided |= suite.provides if final_status == :success

  Status.new(suite, final_status)
end

#run_test(assertion, setup) ⇒ Object

Formatter callback. Invoked once for every variation of an assertion. Gets the assertion to run as single argument.



205
206
207
208
209
210
211
 
# File 'lib/baretest/run.rb', line 205

def run_test(assertion, setup)
  rv = assertion.execute(setup, assertion.suite.ancestry_teardown)
  @count[:test]     += 1
  @count[rv.status] += 1

  rv
end

#run_test_variants(assertion) ⇒ Object

Invoked once for every assertion. Iterates over all variants of an assertion and invokes run_test for each.



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
 
# File 'lib/baretest/run.rb', line 179

def run_test_variants(assertion)
  ignored = !@states.include?(@last_run_states[assertion.id])
  skipped = @skipped[assertion] || assertion.skipped?

  if ignored then
    overall_status = nil
  elsif skipped then
    Array.new(assertion.suite.component_variant_count) { run_test(assertion, []) }
    @last_run_states[assertion.id] = :manually_skipped
    overall_status                 = :manually_skipped
  else
    states = []
    assertion.suite.each_component_variant do |setups|
      rv = run_test(assertion, setups)
      states << rv.status
    end
    overall_status                 = BareTest.most_important_status(states)
  end
  @last_run_states[assertion.id] = overall_status if overall_status

  overall_status
end

#skip_recursively(suite, reason) ⇒ Object

Marks all tests, suites and their subsuites within this suite as skipped.



223
224
225
226
227
228
 
# File 'lib/baretest/run.rb', line 223

def skip_recursively(suite, reason) # :nodoc:
  skip_suite(suite, reason)
  suite.suites.each do |description, subsuite|
    skip_recursively(subsuite, reason)
  end
end

#skip_suite(suite, reason) ⇒ Object

Marks all assertion within this suite as skipped and the suite itself too.



214
215
216
217
218
219
220
 
# File 'lib/baretest/run.rb', line 214

def skip_suite(suite, reason) # :nodoc:
  suite.skip(reason)
  reason = suite.reason
  suite.assertions.each do |test|
    test.skip(reason)
  end
end