Class: Minitest::Heat::Issue

Inherits:
Object
  • Object
show all
Extended by:
Forwardable
Defined in:
lib/minitest/heat/issue.rb

Overview

Wrapper for Result to provide a more natural-language approach to result details

Constant Summary collapse

TYPES = 
%i[error broken failure skipped painful slow].freeze

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(assertions: 1, test_location: ['Unrecognized Test File', 1], backtrace: [], execution_time: 0.0, message: nil, test_class: nil, test_identifier: nil, passed: false, error: false, skipped: false) ⇒ type

Creates an instance of Issue. In general, the ‘from_result` approach will be more convenient

for standard usage, but for lower-level purposes like testing, the initializer provides3
more fine-grained control

Parameters:

  • assertions: (defaults to: 1) —

    1 [Integer] the number of assertions in the result

  • message: (defaults to: nil) —

    nil [String] exception if there is one

  • backtrace: (defaults to: []) —
    Array<String>

    the array of backtrace lines from an exception

  • test_location: (defaults to: ['Unrecognized Test File', 1]) —

    nil [Array<String, Integer>] the locations identifier for a test

  • test_class: (defaults to: nil) —

    nil [String] the class name for the test result’s containing class

  • test_identifier: (defaults to: nil) —

    nil [String] the name of the test

  • execution_time: (defaults to: 0.0) —

    nil [Float] the time it took to run the test

  • passed: (defaults to: false) —

    false [Boolean] true if the test explicitly passed, false otherwise

  • error: (defaults to: false) —

    false [Boolean] true if the test raised an exception

  • skipped: (defaults to: false) —

    false [Boolean] true if the test was skipped



69
70
71
72
73
74
75
76
77
78
79
80
81
82
 
# File 'lib/minitest/heat/issue.rb', line 69

def initialize(assertions: 1, test_location: ['Unrecognized Test File', 1], backtrace: [], execution_time: 0.0, message: nil, test_class: nil, test_identifier: nil, passed: false, error: false, skipped: false)
  @message = message

  @assertions = Integer(assertions)
  @locations = Locations.new(test_location, backtrace)

  @test_class = test_class
  @test_identifier = test_identifier
  @execution_time = Float(execution_time)

  @passed = passed
  @error = error
  @skipped = skipped
end

Instance Attribute Details

#assertions ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def assertions
  @assertions
end

#error ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def error
  @error
end

#execution_time ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def execution_time
  @execution_time
end

#locations ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def locations
  @locations
end

#message ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def message
  @message
end

#passed ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def passed
  @passed
end

#skipped ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def skipped
  @skipped
end

#test_class ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def test_class
  @test_class
end

#test_identifier ⇒ Object (readonly)

# Long-term, these could be configurable so that people can determine their own thresholds of # pain for slow tests SLOW_THRESHOLDS = 

slow: 1.0,
painful: 3.0

.freeze



20
21
22
 
# File 'lib/minitest/heat/issue.rb', line 20

def test_identifier
  @test_identifier
end

Class Method Details

.from_result(result) ⇒ Issue

Extracts the necessary data from result.

Parameters:

  • result (Minitest::Result) —

    the instance of Minitest::Result to examine

Returns:

  • (Issue) —

    the instance of the issue to use for examining the result



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'lib/minitest/heat/issue.rb', line 36

def self.from_result(result)
  # Not all results are failures, so we use the safe navigation operator
  exception = result.failure&.exception

  new(
    assertions: result.assertions,
    test_location: result.source_location,
    test_class: result.klass,
    test_identifier: result.name,
    execution_time: result.time,
    passed: result.passed?,
    error: result.error?,
    skipped: result.skipped?,
    message: exception&.message,
    backtrace: exception&.backtrace
  )
end

Instance Method Details

#error? ⇒ Boolean

Was there an exception that triggered a failure?

Returns:

  • (Boolean) —

    true if there’s an exception



177
178
179
 
# File 'lib/minitest/heat/issue.rb', line 177

def error?
  error
end

#exception_message_limit ⇒ Object 



208
209
210
 
# File 'lib/minitest/heat/issue.rb', line 208

def exception_message_limit
  200
end

#first_line_of_exception_message ⇒ String

Returns the first line of an exception message when the issue is from a proper exception

failure since exception messages can be long and cumbersome.

Returns:

  • (String) —

    the first line of the exception message



202
203
204
205
206
 
# File 'lib/minitest/heat/issue.rb', line 202

def first_line_of_exception_message
  text = message.split("\n")[0]

  text.size > exception_message_limit ? "#{text[0..exception_message_limit]}..." : text
end

#hit? ⇒ Boolean

Determines if the issue is a proper ‘hit’ which is anything that doesn’t pass or is slow.

(Because slow tests still pass and wouldn't otherwise be considered an issue.)

Returns:

  • (Boolean) —

    true if the test did not pass or if it was slow



115
116
117
 
# File 'lib/minitest/heat/issue.rb', line 115

def hit?
  !passed? || slow? || painful?
end

#in_source? ⇒ Boolean

Determines if the issue is an exception that was raised from directly within the project

codebase.

Returns:

  • (Boolean) —

    true if the final locations of the stacktrace was a file from the project (as opposed to a dependency or Ruby library)



162
163
164
 
# File 'lib/minitest/heat/issue.rb', line 162

def in_source?
  locations.proper_failure?
end

#in_test? ⇒ Boolean

Determines if the issue is an exception that was raised from directly within a test

definition. In these cases, it's more likely to be a quick fix.

Returns:

  • (Boolean) —

    true if the final locations of the stacktrace was a test file



153
154
155
 
# File 'lib/minitest/heat/issue.rb', line 153

def in_test?
  locations.broken_test?
end

#painful? ⇒ Boolean

Determines if a test should be considered painfully slow by comparing it to the high end

definition of what is considered slow.

Returns:

  • (Boolean) —

    true if the test took longer to run than ‘painfully_slow_threshold`



145
146
147
 
# File 'lib/minitest/heat/issue.rb', line 145

def painful?
  execution_time >= painfully_slow_threshold
end

#painfully_slow_threshold ⇒ Float

The number, in seconds, for a test to be considered “painfully slow”

Returns:

  • (Float) —

    number of seconds after which a test is considered painfully slow



129
130
131
 
# File 'lib/minitest/heat/issue.rb', line 129

def painfully_slow_threshold
  Minitest::Heat.configuration.painfully_slow_threshold
end

#passed? ⇒ Boolean

Was the result a pass? i.e. Skips aren’t passes or failures. Slows are still passes. So this

is purely a measure of whether the test explicitly passed all assertions

Returns:

  • (Boolean) —

    false for errors, failures, or skips, true for passes (including slows)



170
171
172
 
# File 'lib/minitest/heat/issue.rb', line 170

def passed?
  passed
end

#skipped? ⇒ Boolean

Was the test skipped?

Returns:

  • (Boolean) —

    true if the test was explicitly skipped, false otherwise



184
185
186
 
# File 'lib/minitest/heat/issue.rb', line 184

def skipped?
  skipped
end

#slow? ⇒ Boolean

Determines if a test should be considered slow by comparing it to the low end definition of

what is considered slow.

Returns:

  • (Boolean) —

    true if the test took longer to run than ‘slow_threshold`



137
138
139
 
# File 'lib/minitest/heat/issue.rb', line 137

def slow?
  execution_time >= slow_threshold && execution_time < painfully_slow_threshold
end

#slow_threshold ⇒ Float

The number, in seconds, for a test to be considered “slow”

Returns:

  • (Float) —

    number of seconds after which a test is considered slow



122
123
124
 
# File 'lib/minitest/heat/issue.rb', line 122

def slow_threshold
  Minitest::Heat.configuration.slow_threshold
end

#summary ⇒ String

The more nuanced detail of the failure. If it’s an error, digs into the exception. Otherwise

uses the message from the result

Returns:

  • (String) —

    a more detailed explanation of the issue



192
193
194
195
196
 
# File 'lib/minitest/heat/issue.rb', line 192

def summary
  # When there's an exception, use the first line from the exception message. Otherwise,  the
  #   message represents explanation for a test failure, and should be used in full
  error? ? first_line_of_exception_message : message
end

#type ⇒ Symbol

Classifies different issue types so they can be categorized, organized, and prioritized.

Primarily helps add some nuance to issue types. For example, an exception that arises from
the project's source code is a genuine exception. But if the exception arose directly from
the test, then it's more likely that there's just a simple syntax issue in the test.
Similarly, the difference between a moderately slow test and a painfully slow test can be
significant. A test that takes half a second is slow, but a test that takes 10 seconds is
painfully slow and should get more attention.

Returns:

  • (Symbol) —

    issue type for classifying issues and reporting



93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/minitest/heat/issue.rb', line 93

def type # rubocop:disable Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
  if error? && in_test?
    :broken
  elsif error?
    :error
  elsif skipped?
    :skipped
  elsif !passed?
    :failure
  elsif passed? && painful?
    :painful
  elsif passed? && slow?
    :slow
  else
    :success
  end
end