Module: Whitestone

Included in:
Object
Defined in:
lib/whitestone/assertion_classes.rb,
lib/whitestone.rb,
lib/whitestone/output.rb,
lib/whitestone/version.rb,
lib/whitestone/custom_assertions.rb

Overview

————————————————————————— #

Defined Under Namespace

Modules: Assertion Classes: AssertionSpecificationError, ErrorOccurred, FailureOccurred, Output, Scope, Test

Constant Summary collapse

ASSERTION_CLASSES =

^^^ Assertion::Custom

{
  :T =>  Assertion::True,       :F =>  Assertion::False,  :N => Assertion::Nil,
  :Eq => Assertion::Equality,   :Mt => Assertion::Match,  :Ko => Assertion::KindOf,
  :Ft => Assertion::FloatEqual, :Id => Assertion::Identity,
  :E =>  Assertion::ExpectError, :C => Assertion::Catch,
  :custom => Assertion::Custom
}
D =

Allows before and after hooks to be specified via the following method syntax when this module is mixed-in:

D .<< { puts "before all nested tests" }
D .<  { puts "before each nested test" }
D .>  { puts "after  each nested test" }
D .>> { puts "after  all nested tests" }

::Whitestone
VERSION = 
"1.0.2"

Class Attribute Summary collapse

Class Method Summary collapse

Class Attribute Details

.caught_value ⇒ Object

When a C assertion is run (i.e. that the expected symbol will be thrown), the value that is thrown along with the symbol will be stored in Whitestone.caught_value in case it needs to be tested. If no value is thrown, this accessor will contain nil.



117
118
119
 
# File 'lib/whitestone.rb', line 117

def caught_value
  @caught_value
end

.exception ⇒ Object

When an E assertion is run (i.e. that the expected error will be raised), the exception that is rescued will be stored in Whitestone.exception in case it needs to be tested.



123
124
125
 
# File 'lib/whitestone.rb', line 123

def exception
  @exception
end

.stats ⇒ Object (readonly)

‘stats’ is a hash with the following keys:

:pass   :fail   :error   :assertions   :time



102
103
104
 
# File 'lib/whitestone.rb', line 102

def stats
  @stats
end

Class Method Details

.<(*args, &block) ⇒ Object

Registers the given block to be executed before each nested test inside this test.



165
166
167
168
169
170
171
172
173
 
# File 'lib/whitestone.rb', line 165

def <(*args, &block)
  if args.empty?
    raise ArgumentError, 'block must be given' unless block
    @current_scope.before_each << block
  else
    # the < method is being used as a check for inheritance
    super
  end
end

.<<(&block) ⇒ Object

Registers the given block to be executed before all nested tests inside this test.

Raises:

  • (ArgumentError) 


184
185
186
187
 
# File 'lib/whitestone.rb', line 184

def << &block
  raise ArgumentError, 'block must be given' unless block
  @current_scope.before_all << block
end

.>(&block) ⇒ Object

Registers the given block to be executed after each nested test inside this test.

Raises:

  • (ArgumentError) 


177
178
179
180
 
# File 'lib/whitestone.rb', line 177

def > &block
  raise ArgumentError, 'block must be given' unless block
  @current_scope.after_each << block
end

.>>(&block) ⇒ Object

Registers the given block to be executed after all nested tests inside this test.

Raises:

  • (ArgumentError) 


191
192
193
194
 
# File 'lib/whitestone.rb', line 191

def >> &block
  raise ArgumentError, 'block must be given' unless block
  @current_scope.after_all << block
end

.action(base, assert_negate_query, *args, &block) ⇒ Object

Whitestone.action

This is an absolutely key method. It implements T, F, Eq, T!, F?, Eq?, etc. After some sanity checking, it creates an assertion object, runs it, and sees whether it passed or failed.

If the assertion fails, we raise FailureOccurred, with the necessary information about the failure. If an error happens while the assertion is run, we don’t catch it. Both the error and the failure are handled upstream, in Whitestone.call.

It’s worth noting that errors can occur while tests are run that are unconnected to this method. Consider these two examples:

T { "foo".frobnosticate? }     -- error occurs on our watch
T "foo".frobnosticate?         -- error occurs before T() is called

By letting errors from here escape, the two cases can be dealt with together.

T and F are special cases: they can be called with custom assertions.

T :circle, c, [4,1, 10, :H]
  -> run_custom_test(:circle, :assert, [4,1,10,:H])



317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
 
# File 'lib/whitestone.rb', line 317

def action(base, assert_negate_query, *args, &block)
  mode = assert_negate_query    # :assert, :negate or :query

  # Sanity checks: these should never fail!
  unless [:assert, :negate, :query].include? mode
    raise AssertionSpecificationError, "Invalid mode: #{mode.inspect}"
  end
  unless ASSERTION_CLASSES.key? base
    raise AssertionSpecificationError, "Invalid base: #{base.inspect}"
  end

  # Special case: T may be used to invoke custom assertions.
  # We catch the use of F as well, even though it's disallowed, so that
  # we can give an appropriate error message.
  if base == :T or base == :F and args.size > 1 and args.first.is_a? Symbol
    if base == :T and mode == :assert
      # Run a custom assertion.
      inside_custom_assertion do
        action(:custom, :assert, *args)
      end
      return nil
    else
      message =  "You are attempting to run a custom assertion.\n"
      message << "These can only be run with T, not F, T?, T!, F? etc."
      raise AssertionSpecificationError, message
    end
  end

  assertion = ASSERTION_CLASSES[base].new(mode, *args, &block)
    # e.g. assertion = Assertion::Equality(:assert, 4, 4)   # no block
    #      assertion = Assertion::Nil(:query) { names.find "Tobias" }
    #      assertion = Assertion::Custom(...)

  stats[:assertions] += 1 unless @inside_custom_assertion

  # We run the assertion (returns true for pass and false for fail).
  passed = assertion.run

  # We negate the result if neccesary...
  case mode
  when :negate then passed = ! passed
  when :query  then return passed
  end
  # ...and report a failure if necessary.
  if passed
    # We do this here because we only want the test to pass if it actually
    # runs an assertion; otherwise its result is 'blank'.  If a later
    # assertion in the test fails or errors, the result will be rewritten.
    @current_test.result = :pass if @current_test
  else
    calling_context = assertion.block || @calls.last
    backtrace = caller
    raise FailureOccurred.new(calling_context, assertion.message, backtrace)
  end
end

.call(block, sandbox = nil) ⇒ Object

Whitestone.call

Invokes the given block and debugs any exceptions that may arise as a result. The block can be from a Test object or a “before-each”-style block.

If an assertion fails or an error occurs during the running of a test, it is dealt with in this method (update the stats, update the test object, re-raise so the upstream method execute can abort the current test/scope.



601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
 
# File 'lib/whitestone.rb', line 601

def call(block, sandbox = nil)
  begin
    @calls.push block

    if sandbox
      sandbox.instance_eval(&block)
    else
      block.call
    end

  rescue FailureOccurred => f
    ## A failure has occurred while running a test.  We report the failure
    ## and re-raise the exception so that the calling code knows not to
    ## continue with this test.
    @stats[:fail] += 1
    @current_test.result = :fail
    @output.report_failure( current_test, f.message, f.backtrace )
    raise

  rescue Exception, AssertionSpecificationError => e
    ## An error has occurred while running a test.
    ##   OR
    ## An assertion was not properly specified.
    ##
    ## We record and report the error and then raise Whitestone::ErrorOccurred
    ## so that the code running the test knows an error occurred.  It
    ## doesn't need to do anything with the error; it's just a signal.
    @stats[:error] += 1
    @current_test.result = :error
    @current_test.error  = e
    if e.class == AssertionSpecificationError
      @output.report_uncaught_exception( current_test, e, @calls, :filter )
    else
      @output.report_uncaught_exception( current_test, e, @calls )
    end
    raise ErrorOccurred

  ensure
    @calls.pop
  end
end

.create_test(insulate, *description, &block) ⇒ Object

Raises:

  • (ArgumentError) 


153
154
155
156
157
158
159
160
 
# File 'lib/whitestone.rb', line 153

def create_test insulate, *description, &block
  raise ArgumentError, 'block must be given' unless block
  description = description.join(' ')
  sandbox = Object.new if insulate
  new_test = Whitestone::Test.new(description, block, sandbox)
  new_test.parent = @tests.last
  @current_scope.tests << new_test
end

.current_test ⇒ Object

The description of the currently-running test. Very useful for conditional breakpoints in library code. E.g.

debugger if Whitestone.current_test =~ /something.../



108
109
110
 
# File 'lib/whitestone.rb', line 108

def current_test
  (@current_test.nil?) ? "(toplevel)" : @current_test.description
end

.custom(name, definition) ⇒ Object

Whitestone.custom defines a custom assertion.

Example usage:

Whitestone.custom :circle, {
  :description => "Circle equality",
  :parameters  => [ [:circle, Circle], [:values, Array] ],
  :run => lambda { |circle, values|
    x, y, r, label = values
    test('x')     { Ft x, circle.centre.x         }
    test('y')     { Ft y, circle.centre.y         }
    test('r')     { Ft r, circle.radius           }
    test('label') { Eq Label[label], circle.label }
  }
}



403
404
405
 
# File 'lib/whitestone.rb', line 403

def custom(name, definition)
  define_custom_assertion(name, definition)
end

.D(*description, &block) ⇒ Object

Defines a new test composed of the given description and the given block to execute.

This test may contain nested tests.

Tests at the outer-most level are automatically insulated from the top-level Ruby environment.



140
141
142
 
# File 'lib/whitestone.rb', line 140

def D *description, &block
  create_test @tests.empty?, *description, &block
end

.D!(*description, &block) ⇒ Object

Defines a new test that is explicitly insulated from the tests that contain it and also from the top-level Ruby environment.

This test may contain nested tests.



149
150
151
 
# File 'lib/whitestone.rb', line 149

def D! *description, &block
  create_test true, *description, &block
end

.define_custom_assertion(name, definition) ⇒ Object 



407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/whitestone.rb', line 407

def define_custom_assertion(name, definition)
  legitimate_keys = Set[:description, :parameters, :check, :run]
  unless Symbol === name and Hash === definition and
         (definition.keys + [:check]).all? { |key| legitimate_keys.include? key }
    message = %{
      #
      #Usage:
      #  Whitestone.custom(name, definition)
      #      where name is a symbol
      #        and definition is a hash with keys :description, :parameters, :run
      #                                           and optionally :check
    }.___margin
    raise AssertionSpecificationError, Col[message].yb
  end
  Assertion::Custom.define(name, definition)
end

.execute ⇒ Object

Whitestone.execute

Executes the current test scope recursively. A SCOPE is a collection of D blocks, and the contents of each D block is a TEST, comprising a description and a block of code. Because a test block may contain D statements within it, when a test block is run @current_scope is set to Scope.new so that newly-encountered tests can be added to it. That scope is then executed recursively. The invariant is this: @current_scope is the CURRENT scope to which tests may be added. At the end of ‘execute’, The per-test guts of this method have been extracted to execute_test so that the structure of execute is easier to see. execute_test contains lots of exception handling and comments.



521
522
523
524
525
526
527
528
529
530
531
532
533
 
# File 'lib/whitestone.rb', line 521

def execute
  @current_scope.before_all.each {|b| call b }     # Run pre-test setup
  @current_scope.tests.each do |test|              # Loop through tests
    @current_scope.before_each.each {|b| call b }  # Run per-test setup
    @tests.push test; @current_test = test

    execute_test(test)                             # Run the test

    @tests.pop; @current_test = @tests.last
    @current_scope.after_each.each {|b| call b }   # Run per-test teardown
  end
  @current_scope.after_all.each {|b| call b }      # Run post-test teardown
end

.execute_test(test) ⇒ Object

Whitestone.execute_test

Executes a single test (block containing assertions). That wouldn’t be so hard, except that there could be new tests defined within that block, so we need to create a new scope into which such tests may be placed [in create_test – << Test.new(…)].

The old scope is restored at the end of the method.

The new scope is executed recursively in order to run any tests created therein.

Exception (and failure) handling is straightforward here. The hard work is done in call; we just catch them and do nothing. The point is to avoid the recursive execute: fail fast.



553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
 
# File 'lib/whitestone.rb', line 553

def execute_test(test)
  stored_scope = @current_scope
  begin
    # Create nested scope in case a 'D' is encountered while running the test.
    @current_scope = Whitestone::Scope.new

    # Run the test block, which may create new tests along the way (if the
    # block includes any calls to 'D').
    call test.block, test.sandbox

    # Increment the pass count _if_ the current test passed, which it only
    # does if at least one assertion was run.
    @stats[:pass] += 1 if @current_test.passed?

    # Execute the nested scope.  Nothing will happen if there are no tests
    # in the nested scope because before_all, tests and after_all will be
    # empty.
    execute

  rescue FailureOccurred => f
    # See method-level comment regarding exception handling.
    :noop
  rescue ErrorOccurred => e
    :noop
  rescue Exception => e
    # We absolutely should not be receiving an exception here.  Exceptions
    # are caught up the line, dealt with, and ErrorOccurred is raised.  If
    # we get here, something is strange and we should exit.
    STDERR.puts "Internal error: #{__FILE__}:#{__LINE__}; exiting"
    puts e.inspect
    puts e.backtrace
    exit!
  ensure
    # Restore the previous values of @current_scope
    @current_scope = stored_scope
  end
end

.inside_custom_assertion ⇒ Object

inside_custom_assertion allows us (via yield) to run a custom assertion without racking up the assertion count for each of the assertions therein. Todo: consider making it a stack so that custom assertions can be nested.



379
380
381
382
383
384
385
 
# File 'lib/whitestone.rb', line 379

def inside_custom_assertion
  @inside_custom_assertion = true
  stats[:assertions] += 1
  yield
ensure
  @inside_custom_assertion = false
end

.record_execution_time ⇒ Object

Record the elapsed time to execute the given block.



498
499
500
501
502
503
 
# File 'lib/whitestone.rb', line 498

def record_execution_time
  start = Time.now
  yield
  finish = Time.now
  finish - start
end

.run(options = {}) ⇒ Object

Whitestone.run

Executes all tests defined thus far. Tests are defined by ‘D’ blocks. Test objects live in a Scope. @current_scope is the top-level scope, but this variable is changed during execution to point to nested scopes as needed (and then changed back again).

This method should therefore be run after all the tests have been defined, e.g. in an at_exit clause. Requiring ‘whitestone/auto’ does that for you.

Argument: options hash

  • :filter is a Regex. Only top-level tests whose descriptions match that regex will be run.

  • :full_backtrace is true or false: do you want the backtraces reported in event of failure or error to be filtered or not? Most of the time you would want them to be filtered (therefore false).



454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
 
# File 'lib/whitestone.rb', line 454

def run(options={})
  test_filter_pattern = options[:filter]
  @output.set_full_backtrace if options[:full_backtrace]
  # Clear previous results.
  @stats.clear
  @tests.clear

  # Filter the tests if asked to.
  if test_filter_pattern
    @top_level.filter(test_filter_pattern)
    if @top_level.tests.empty?
      msg = "!! Applied filter #{test_filter_pattern.inspect}, which left no tests to be run!"
      STDERR.puts Col[msg].yb
      exit
    end
  end

  # Execute the tests.
  @stats[:time] = record_execution_time do
    catch(:stop_dfect_execution) do
      execute       # <-- This is where the real action takes place.
    end
  end

  # Display reports.
  @output.display_test_by_test_result(@top_level)
  @output.display_details_of_failures_and_errors
  @output.display_results_npass_nfail_nerror_etc(@stats)

  @top_level = @current_scope = Whitestone::Scope.new
  # ^^^ In case 'run' gets called again; we don't want to re-run the old tests.
end

.S(identifier, &block) ⇒ Object

Mechanism for sharing code between tests.

S :values do
  @values = [8,9,10]
end

D "some test" do
  S :values
  Eq @values.last, 10
end



207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
 
# File 'lib/whitestone.rb', line 207

def S identifier, &block
  if block_given?
    if already_shared = @share[identifier]
      msg = "A code block #{already_shared.inspect} has already " \
            "been shared under the identifier #{identifier.inspect}."
      raise ArgumentError, msg
    end
    @share[identifier] = block

  elsif block = @share[identifier]
    if @tests.empty?
      msg = "Cannot inject code block #{block.inspect} shared under " \
            "identifier #{identifier.inspect} outside of a Whitestone test."
      raise 
    else
      # Find the closest insulated parent test; this should always
      # succeed because root-level tests are insulated by default.
      test = @tests.reverse.find { |t| t.sandbox }
      test.sandbox.instance_eval(&block)
    end

  else
    raise ArgumentError, "No code block is shared under " \
                         "identifier #{identifier.inspect}."
  end
end

.S!(identifier, &block) ⇒ Object

Shares the given code block AND inserts it in-place. (Well, by in-place, I mean the closest insulated block.)



236
237
238
239
240
 
# File 'lib/whitestone.rb', line 236

def S! identifier, &block
  raise 'block must be given' unless block_given?
  S identifier, &block
  S identifier
end

.S?(identifier) ⇒ Boolean

Checks whether any code has been shared under the given identifier.

Returns:

  • (Boolean) 


243
244
245
 
# File 'lib/whitestone.rb', line 243

def S? identifier
  @share.key? identifier
end

.stop ⇒ Object

Whitestone.stop

Stops the execution of the run method or raises an exception if that method is not currently executing.



493
494
495
 
# File 'lib/whitestone.rb', line 493

def stop
  throw :stop_dfect_execution
end