ABAnalyzer

ABAnalyzer is a Ruby library that will perform testing to determine if there is a statistical difference in categorical data (typically called an A/B Test). By default, it uses a G-Test for independence, but a Chi-Square test for independence can also be used. For more information, see:

• en.wikipedia.org/wiki/G-test

• en.wikipedia.org/wiki/Pearson%27s_chi-square_test

Installation

Simply run:

gem install abanalyzer

Basic Usage

The simplest test (which uses a gtest):

require 'rubygems'
require 'abanalyzer'

values = {}
values[:agroup] = { :opened => 100, :unopened => 300 }
values[:bgroup] = { :opened => 50, :unopened => 350 }

tester = ABAnalyzer::ABTest.new values
# Are the two different?  Returns true or false (at 0.05 level of significance)
puts tester.different?

Multiple Categories

You can use the ABAnalyzer module to test for differences in more than two categories. For instance, to test accross three:

require 'rubygems'
require 'abanalyzer'

values = {}
values[:agroup] = { :male => 200, :female => 250 }
values[:bgroup] = { :male => 150, :female => 300}
values[:cgroup] = { :male => 50, :female => 50 }

tester = ABAnalyzer::ABTest.new values
# Are the two different?  Returns true or false (at 0.05 level of significance)
puts tester.different?

Tests Available

You can get the actual p-value for either a Chi-Square test for independence or a G-Test for independence.

...
tester = ABAnalyzer::ABTest.new values
puts tester.chisquare_p
puts tester.gtest_p

You can additionally get the actual score for either a Chi-Square test for independence or a G-Test for independence.

...
tester = ABAnalyzer::ABTest.new values
puts tester.chisquare_score
puts tester.gtest_score

Running Tests

Testing can be run by using:

rake test