Saber - The Simple A/B Experiment Renderer

What is Saber?

Saber provides a very thin layer for organizing and displaying a/b or "split" test experiments. It is designed to facilitate a/b testing of site designs, experiences and workflows, but is meant as a lightweight alternatives to other a/b test frameworks.

Split is not a full-featured framework. It has one role, which is to provide a simple DSL for displaying variations in a/b exeperiments. It is not responsible for recording those experiments. In this way we avoid coupling to specific database implementations.

There are many, many very good analytics tools and frameworks out there, and we feel those are going to be the best suited to recording and analyzing the results of a/b experiments.

How to use Saber

In the Controllers

Saber adds a simple helper method to setup an A/B experiment to your controllers

For example to setup an A/B test with three variants and equal weighting:

def index
  setup_ab_test_parameters ['v1', 'v2', 'v3']
end

If some non-uniform weighting is desired simply use a Hash

def index
  setup_ab_test_parameters { v1: 1, v2: 3, v3:1 }
end

In the above example 'v2' is 3 times more likely than v1 or v3 to be shown. The weighting can be any numbers including floats, or percent values, they are simply used relative to their total.

In the Views

Saber simply sets a parameter called @ab_version which is available in the view. However a much nicer handy view helper is provided in stead. The recommended usage is:

-if ab_version(:v1)
  -# v1 specific view content goes here
-elsif ab_version(:v2)
  -# v2 specific view content goes here

There are a number of ways you can organize content for A/B tests which is why Saber takes no opinions on this matter. Depending on the type of A/B test the approach you may want to take can vary greatly.

If the content between tests varies greatly then I recommend using a seperate partial for each test and then use the view helper to add the conditional logic to decide which partial to render.

If the content differences are small then just sprinkle where needed.

Ensuring consistent testing using session variables

Saber stores a session value in session[:page_version] with the name of the ab-test version. It ensures that repeated views by the same person show the same page (showing a different page each time would be strange and also probably invalidate you a/b test results somewhat).

Overriding Saber

Saber can also be overidden as needed by passing a query parameter like ?page_version=v2. This will skip the random selection (or the current session parameter) and show the 'v2' page every time (it also sets session[:page_version] to 'v2' for convenience).

Where are the Analytics?

There are so many great analytics platforms out there for collecting a/b testing metrics that do a better job than I (or most people) could hope to achieve with a simple gem.

This means you are free to analyse your a/b test results with any one you choose (I like Mixpanel myself). Saber may provide some integration helpers for these services in the future (pull requests accepted).

If you really want to host your analytics then Split is probably much better suited to your needs.

This projectd uses the MIT-LICENSE.