Module: Abstats

Defined in:
lib/abstats/abstats.rb,
lib/abstats/version.rb

Constant Summary collapse

VERSION = 
"0.0.1"

Class Method Summary collapse

Class Method Details

.statistical_power(n_baseline, p_baseline, n_exp, p_exp, confidence = 0.95) ⇒ Object

Give the statistical power representing the probability that the change seen in the experiment is in fact real. More specifically, the null hypothesis is that the experiment and the baseline are identical, this returns the probability that the test will reject the null hypothesis.



8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
 
# File 'lib/abstats/abstats.rb', line 8

def self.statistical_power(n_baseline, p_baseline, n_exp, p_exp, confidence=0.95)
  if(p_baseline == p_exp or p_exp == 0 or p_baseline == 0)
    return 0
  end

  z = Statistics2.pnormaldist(confidence + ((1 - confidence)/2))
  
  p = (p_baseline + p_exp) / 2
  q = ((1 - p_baseline) + (1 - p_exp)) / 2

  variance_baseline = p_baseline * (1 - p_baseline)
  variance_exp = p_exp * (1 - p_exp)

  n = [n_baseline, n_exp].min.to_i

  a = Math.sqrt(n) * (p_baseline - p_exp).abs
  b = z * Math.sqrt(2.0 * p * q)
  c = Math.sqrt(variance_baseline + variance_exp)
  
  lower = Statistics2.normaldist((a - b) / c)
  upper = Statistics2.normaldist((a + b) / c)
  
  lower + (1.0 - upper)
end