Module: Benchmark::Compare

Included in:
Benchmark
Defined in:
lib/benchmark/compare.rb

Overview

Functionality of performaing comparison between reports.

Usage:

Add x.compare! to perform comparison between reports.

Example:

> Benchmark.ips do |x|
x.report('Reduce using tag')     { [*1..10].reduce(:+) }
x.report('Reduce using to_proc') { [*1..10].reduce(&:+) }
x.compare!
end

Calculating -------------------------------------
  Reduce using tag     19216 i/100ms
Reduce using to_proc     17437 i/100ms
-------------------------------------------------
  Reduce using tag   278950.0 (±8.5%) i/s -    1402768 in   5.065112s
Reduce using to_proc   247295.4 (±8.0%) i/s -    1238027 in   5.037299s

Comparison:
  Reduce using tag:   278950.0 i/s
Reduce using to_proc:   247295.4 i/s - 1.13x slower

Besides regular Calculating report, this will also indicates which one is slower.

x.compare! also takes an order: :baseline option.

Example:

Benchmark.ips do |x|

x.report('Reduce using block') { [*1..10].reduce { |sum, n| sum + n } } x.report('Reduce using tag') { [*1..10].reduce(:+) } x.report('Reduce using to_proc') { [*1..10].reduce(&:+) } x.compare!(order: :baseline) end

Calculating ------------------------------------- Reduce using block 886.202k (± 2.2%) i/s - 4.521M in 5.103774s Reduce using tag 1.821M (± 1.6%) i/s - 9.111M in 5.004183s Reduce using to_proc 895.948k (± 1.6%) i/s - 4.528M in 5.055368s

Comparison:

Reduce using block:   886202.5 i/s
Reduce using tag:  1821055.0 i/s - 2.05x  (± 0.00) faster

Reduce using to_proc: 895948.1 i/s - same-ish: difference falls within error

The first report is considered the baseline against which other reports are compared.

Instance Method Summary collapse

Instance Method Details

#compare(*entries, order: :fastest) ⇒ Object

Compare between reports, prints out facts of each report: runtime, comparative speed difference.

Parameters:

  • entries (Array<Report::Entry>) —

    Reports to compare.



56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'lib/benchmark/compare.rb', line 56

def compare(*entries, order: :fastest)
  return if entries.size < 2

  max_width = entries.map { |e| e.label.to_s.size }.max

  case order
  when :baseline
    baseline = entries.shift
    sorted = entries.sort_by{ |e| e.stats.central_tendency }.reverse
  when :fastest
    sorted = entries.sort_by{ |e| e.stats.central_tendency }.reverse
    baseline = sorted.shift
  else
    raise ArgumentError, "Unknown order: #{order.inspect}"
  end

  $stdout.puts "\nComparison:"

  $stdout.printf "%#{max_width}s: %10.1f i/s\n", baseline.label.to_s, baseline.stats.central_tendency

  sorted.each do |report|
    name = report.label.to_s

    $stdout.printf "%#{max_width}s: %10.1f i/s - ", name, report.stats.central_tendency

    if report.stats.overlaps?(baseline.stats)
      $stdout.print "same-ish: difference falls within error"
    elsif report.stats.central_tendency > baseline.stats.central_tendency
      speedup, error = report.stats.speedup(baseline.stats)
      $stdout.printf "%.2fx ", speedup
      if error
        $stdout.printf " (± %.2f)", error
      end
      $stdout.print " faster"
    else
      slowdown, error = report.stats.slowdown(baseline.stats)
      $stdout.printf "%.2fx ", slowdown
      if error
        $stdout.printf " (± %.2f)", error
      end
      $stdout.print " slower"
    end

    $stdout.puts
  end

  footer = baseline.stats.footer
  $stdout.puts footer.rjust(40) if footer

  $stdout.puts
end