Class: Aggregate
- Inherits:
-
Object
- Object
- Aggregate
- Defined in:
- lib/aggregate.rb
Overview
Implements aggregate statistics and maintains configurable histogram for a set of given samples. Convenient for tracking high throughput data.
Constant Summary collapse
- @@LOG_BUCKETS =
The number of buckets in the binary logarithmic histogram (low => 2**0, high => 2**@@LOG_BUCKETS)
128
Instance Attribute Summary collapse
-
#count ⇒ Object
readonly
The current number of samples.
-
#max ⇒ Object
readonly
The maximum sample value.
-
#min ⇒ Object
readonly
The minimum samples value.
-
#outliers_high ⇒ Object
readonly
The number of samples falling above the highest valued histogram bucket.
-
#outliers_low ⇒ Object
readonly
The number of samples falling below the lowest valued histogram bucket.
-
#sum ⇒ Object
readonly
The sum of all samples.
Instance Method Summary collapse
-
#<<(data) ⇒ Object
Include a sample in the aggregate.
-
#each ⇒ Object
Iterate through each bucket in the histogram regardless of its contents.
-
#each_nonzero ⇒ Object
Iterate through only the buckets in the histogram that contain samples.
-
#initialize(low = nil, high = nil, width = nil) ⇒ Aggregate
constructor
Create a new Aggregate that maintains a binary logarithmic histogram by default.
-
#mean ⇒ Object
The current average of all samples.
-
#skip_row(value_width) ⇒ Object
We denote empty buckets with a ‘~’.
-
#std_dev ⇒ Object
Calculate the standard deviation.
-
#to_s(columns = nil) ⇒ Object
Generate a pretty-printed ASCII representation of the histogram.
Constructor Details
#initialize(low = nil, high = nil, width = nil) ⇒ Aggregate
Create a new Aggregate that maintains a binary logarithmic histogram by default. Specifying values for low, high, and width configures the aggregate to maintain a linear histogram with (high - low)/width buckets
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# File 'lib/aggregate.rb', line 29 def initialize (low=nil, high=nil, width=nil) @count = 0 @sum = 0.0 @sum2 = 0.0 @outliers_low = 0 @outliers_high = 0 # If the user asks we maintain a linear histogram where # values in the range [low, high) are bucketed in multiples # of width if (nil != low && nil != high && nil != width) #Validate linear specification if high <= low raise ArgumentError, "High bucket must be > Low bucket" end if high - low < width raise ArgumentError, "Histogram width must be <= histogram range" end if 0 != (high - low).modulo(width) raise ArgumentError, "Histogram range (high - low) must be a multiple of width" end @low = low @high = high @width = width else @low = 1 @width = nil @high = to_bucket(@@LOG_BUCKETS - 1) end #Initialize all buckets to 0 @buckets = Array.new(bucket_count, 0) end |
Instance Attribute Details
#count ⇒ Object (readonly)
The current number of samples
6 7 8 |
# File 'lib/aggregate.rb', line 6 def count @count end |
#max ⇒ Object (readonly)
The maximum sample value
9 10 11 |
# File 'lib/aggregate.rb', line 9 def max @max end |
#min ⇒ Object (readonly)
The minimum samples value
12 13 14 |
# File 'lib/aggregate.rb', line 12 def min @min end |
#outliers_high ⇒ Object (readonly)
The number of samples falling above the highest valued histogram bucket
21 22 23 |
# File 'lib/aggregate.rb', line 21 def outliers_high @outliers_high end |
#outliers_low ⇒ Object (readonly)
The number of samples falling below the lowest valued histogram bucket
18 19 20 |
# File 'lib/aggregate.rb', line 18 def outliers_low @outliers_low end |
#sum ⇒ Object (readonly)
The sum of all samples
15 16 17 |
# File 'lib/aggregate.rb', line 15 def sum @sum end |
Instance Method Details
#<<(data) ⇒ Object
Include a sample in the aggregate
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
# File 'lib/aggregate.rb', line 68 def << data # Update min/max if 0 == @count @min = data @max = data else @max = data if data > @max @min = data if data < @min end # Update the running info @count += 1 @sum += data @sum2 += (data * data) # Update the bucket @buckets[to_index(data)] += 1 unless outlier?(data) end |
#each ⇒ Object
Iterate through each bucket in the histogram regardless of its contents
187 188 189 190 191 |
# File 'lib/aggregate.rb', line 187 def each @buckets.each_with_index do |count, index| yield(to_bucket(index), count) end end |
#each_nonzero ⇒ Object
Iterate through only the buckets in the histogram that contain samples
195 196 197 198 199 |
# File 'lib/aggregate.rb', line 195 def each_nonzero @buckets.each_with_index do |count, index| yield(to_bucket(index), count) if count != 0 end end |
#mean ⇒ Object
The current average of all samples
89 90 91 |
# File 'lib/aggregate.rb', line 89 def mean @sum / @count end |
#skip_row(value_width) ⇒ Object
We denote empty buckets with a ‘~’
149 150 151 |
# File 'lib/aggregate.rb', line 149 def skip_row(value_width) sprintf("%#{value_width}s ~\n", " ") end |
#std_dev ⇒ Object
Calculate the standard deviation
94 95 96 |
# File 'lib/aggregate.rb', line 94 def std_dev Math.sqrt((@sum2.to_f - ((@sum.to_f * @sum.to_f)/@count.to_f)) / (@count.to_f - 1)) end |
#to_s(columns = nil) ⇒ Object
Generate a pretty-printed ASCII representation of the histogram
107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 |
# File 'lib/aggregate.rb', line 107 def to_s(columns=nil) #default to an 80 column terminal, don't support < 80 for now if nil == columns columns = 80 else raise ArgumentError if columns < 80 end #Find the largest bucket and create an array of the rows we intend to print disp_buckets = Array.new max_count = 0 total = 0 @buckets.each_with_index do |count, idx| next if 0 == count max_count = [max_count, count].max disp_buckets << [idx, to_bucket(idx), count] total += count end #XXX: Better to print just header --> footer return "Empty histogram" if 0 == disp_buckets.length #Figure out how wide the value and count columns need to be based on their #largest respective numbers value_str = "value" count_str = "count" total_str = "Total" value_width = [disp_buckets.last[1].to_s.length, value_str.length].max value_width = [value_width, total_str.length].max count_width = [total.to_s.length, count_str.length].max = columns - (value_width + " |".length + "| ".length + count_width) #Determine the value of a '@' weight = [max_count.to_f/.to_f, 1.0].max #format the header histogram = sprintf("%#{value_width}s |", value_str) .times { histogram << "-"} histogram << sprintf("| %#{count_width}s\n", count_str) # We denote empty buckets with a '~' def skip_row(value_width) sprintf("%#{value_width}s ~\n", " ") end #Loop through each bucket to be displayed and output the correct number prev_index = disp_buckets[0][0] - 1 disp_buckets.each do |x| #Denote skipped empty buckets with a ~ histogram << skip_row(value_width) unless prev_index == x[0] - 1 prev_index = x[0] #Add the value row = sprintf("%#{value_width}d |", x[1]) #Add the bar = (x[2]/weight).to_i .times { row += "@"} ( - ).times { row += " " } #Add the count row << sprintf("| %#{count_width}d\n", x[2]) #Append the finished row onto the histogram histogram << row end #End the table histogram << skip_row(value_width) if disp_buckets.last[0] != bucket_count-1 histogram << sprintf("%#{value_width}s", "Total") histogram << " |" .times {histogram << "-"} histogram << "| " histogram << sprintf("%#{count_width}d\n", total) end |