Class: Leva::BaseEval Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/leva.rb

Overview

This class is abstract.

Subclass and override #evaluate to implement custom evaluation logic.

Base class for all evaluation implementations in Leva.

Instance Method Summary collapse

Instance Method Details

#evaluate(prediction, recordable) ⇒ Float

Evaluates the model’s prediction against the ground truth.

Parameters:

  • prediction (Object) —

    The model’s prediction.

  • recordable (Object) —

    The recordable object containing the ground truth.

Returns:

  • (Float) —

    The evaluation score.

Raises:

  • (NotImplementedError) —

    if the method is not implemented in a subclass.



113
114
115
 
# File 'lib/leva.rb', line 113

def evaluate(prediction, recordable)
  raise NotImplementedError, "#{self.class} must implement #evaluate"
end

#evaluate_and_store(experiment, runner_result) ⇒ Leva::EvaluationResult

Evaluates a single runner result and stores the evaluation.

Parameters:

Returns:



122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/leva.rb', line 122

def evaluate_and_store(experiment, runner_result)
  @experiment = experiment
  @runner_result = runner_result

  score = evaluate(runner_result, runner_result.dataset_record.recordable)

  EvaluationResult.create!(
    experiment: experiment,
    dataset_record: runner_result.dataset_record,
    runner_result: runner_result,
    score: score,
    evaluator_class: self.class.name
  )
end