Class: Kotoshu::ConformanceRunner

Inherits:
Object
  • Object
show all
Defined in:
lib/kotoshu/conformance_runner.rb

Overview

Replays the committed conformance vectors through one backend (or both) and reports differences — the dual-backend suite of plan 66 (TODO.impl 05 P4b).

The vectors (conformance/vectors.jsonl, exported by ConformanceExporter and committed) freeze Ruby engine behavior. This runner replays every vector through a backend and compares the actual against the frozen expectation, so:

  • rake kotoshu:conformance:ruby — the pure-Ruby engine must reproduce the vectors (guards against accidental behavior drift).
  • rake kotoshu:conformance:native — the native engine (NativeBackend via a real Spellchecker with backend "native") must reproduce them too.
  • rake kotoshu:conformance:compare — both engines run side by side and every row must agree: expected == ruby == native. Any difference is a backend divergence; the task exits nonzero.

Everything here runs the real engines over the real fixture dictionaries — no doubles anywhere.

Defined Under Namespace

Classes: CompareResult, Failure, RunResult

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(path: self.class.default_path) ⇒ ConformanceRunner

Returns a new instance of ConformanceRunner.

Parameters:

  • path (String) (defaults to: self.class.default_path) —

    JSONL vectors file to replay



55
56
57
58
 
# File 'lib/kotoshu/conformance_runner.rb', line 55

def initialize(path: self.class.default_path)
  @path = path
  @rows = File.readlines(path, chomp: true).map { |line| JSON.parse(line) }
end

Class Method Details

.default_path ⇒ String

Default vectors path: the committed file at the gem repo root.

Returns:

  • (String) 


30
31
32
 
# File 'lib/kotoshu/conformance_runner.rb', line 30

def self.default_path
  File.join(ConformanceExporter.gem_root, "conformance", "vectors.jsonl")
end

Instance Method Details

#compare ⇒ CompareResult

Run both backends and cross-check every row.

Returns:

Raises:



97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
 
# File 'lib/kotoshu/conformance_runner.rb', line 97

def compare
  ruby_result = run(backend: :ruby)
  native_result = run(backend: :native)

  divergences = @rows.each_index.filter_map do |index|
    ruby_actual = ruby_result.actuals[index]
    native_actual = native_result.actuals[index]
    next if ruby_actual == native_actual

    row = @rows[index]
    Failure.new(index: index, kind: row["kind"], dictionary: row["dictionary"],
                input: row["input"], expected: ruby_actual, actual: native_actual)
  end

  CompareResult.new(ruby: ruby_result, native: native_result, divergences: divergences)
end

#row_count ⇒ Integer

Returns vector count.

Returns:

  • (Integer) —

    vector count



61
62
63
 
# File 'lib/kotoshu/conformance_runner.rb', line 61

def row_count
  @rows.size
end

#run(backend:) ⇒ RunResult

Replay every vector through one backend, comparing against the frozen expectations.

Parameters:

  • backend (String, Symbol) —

    "ruby" or "native"

Returns:

Raises:



72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/kotoshu/conformance_runner.rb', line 72

def run(backend:)
  backend_name = backend.to_s
  if backend_name == "native" && !Kotoshu::Native.available?
    raise Native::Unavailable,
          "native backend requested but the extension is not built " \
          "(rake compile) — cannot run the native conformance suite"
  end

  failures = []
  engines = {}
  actuals = @rows.each_with_index.map do |row, index|
    actual = replay(engine_for(row, backend_name, engines), row)
    if actual != row["expected"]
      failures << Failure.new(index: index, kind: row["kind"], dictionary: row["dictionary"],
                              input: row["input"], expected: row["expected"], actual: actual)
    end
    actual
  end
  RunResult.new(backend: backend_name, row_count: @rows.size, failures: failures, actuals: actuals)
end