Class: ThreeScale::Backend::Experiment

Inherits:
Object
  • Object
show all
Includes:
Scientist::Experiment, Logging
Defined in:
lib/3scale/backend/experiment.rb

Overview

This class allows us to perform experiments using the Scientist gem from GitHub. It is useful for comparing two different behaviors when doing a refactoring. The idea is that we can use the old code in the normal flow of the program and at the same time, we can compare its results and performance against new code. This class notifies the logger if the result of the two fragments of code does not match. Also, it sends their execution time to StatsD.

To use this class, you need to declare the old behavior in a ‘use’ block, and the new one in a ‘try’ block. Also, when instantiating the class, we need to give the experiment a name, and define the % of times that it will run. exp = ThreeScale::Backend::Experiment.new(‘my-experiment’, 50) exp.use { old_method } exp.try { new_method } exp.run

‘run’ returns the result of the block sent to the ‘use’ method. Its return value is the one we should use in our program. On the other hand, ‘try’ will swallow any exception raised inside the block. The operations inside the ‘try’ block should be idempotent. Avoid operations like writing to the DB.

Constant Summary collapse

ResultsMismatch = 
Class.new(StandardError)

Instance Method Summary collapse

Methods included from Logging

enable!, included

Constructor Details

#initialize(name, perc_exec) ⇒ Experiment

Returns a new instance of Experiment.



38
39
40
41
42
 
# File 'lib/3scale/backend/experiment.rb', line 38

def initialize(name, perc_exec)
  @name = name
  @perc_exec = perc_exec
  @base_metric = "backend.#{environment}.experiments.#{name}"
end

Instance Method Details

#enabled? ⇒ Boolean

Returns:

  • (Boolean) 


50
51
52
 
# File 'lib/3scale/backend/experiment.rb', line 50

def enabled?
  RANDOM.rand(100) < perc_exec
end

#publish(result) ⇒ Object 



44
45
46
47
48
 
# File 'lib/3scale/backend/experiment.rb', line 44

def publish(result)
  # This only works for the first candidate (try block)
  send_to_statsd(result.control.duration, result.candidates.first.duration)
  check_mismatch(result)
end