Class: FatCore::Table

Inherits:
Object
  • Object
show all
Defined in:
lib/fat_core/table.rb

Overview

A container for a two-dimensional table. All cells in the table must be a String, a Date, a DateTime, a Bignum (or Integer), a BigDecimal, or a boolean. All columns must be of one of those types or be a string convertible into one of the supported types. It is considered an error if a single column contains cells of different types. Any cell that cannot be parsed as one of the numeric, date, or boolean types will have to_s applied

You can initialize a Table in several ways:

  1. with a Nil, which will return an empty table to which rows or columns can be added later,

  2. with the name of a .csv file,

  3. with the name of an .org file,

  4. with an IO or StringIO object for either type of file, but in that case, you need to specify ‘csv’ or ‘org’ as the second argument to tell it what kind of file format to expect,

  5. with an Array of Arrays,

  6. with an Array of Hashes, all having the same keys, which become the names of the column heads,

  7. with an Array of any objects that respond to .keys and .values methods,

  8. with another Table object.

In the case of an array of arrays, if the second array’s first element is a string that looks like a rule separator, ‘———–’, ‘+———-’, etc., the headers will be taken from the first array. In the case of an array of Hashes or Hash-lime objects, the keys of the hashes will be used as the headers. It is assumed that all the hashes have the same keys.

In the resulting Table, the headers are converted into symbols, with all spaces converted to underscore and everything down-cased. So, the heading, ‘Two Words’ becomes the hash header :two_words.

An entire column can be retrieved by header from a Table, thus, #+BEGIN_EXAMPLE tab = Table.new(“example.org”) tab.avg #+END_EXAMPLE will extract the entire ~:age~ column and compute its average, since Column objects respond to aggregate methods, such as ~sum~, ~min~, ~max~, and ~avg~.

Constant Summary collapse

TYPES = 
%w(NilClass TrueClass FalseClass Date DateTime Numeric String)

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input = nil, ext = '.csv') ⇒ Table

Returns a new instance of Table.


46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/fat_core/table.rb', line 46

def initialize(input = nil, ext = '.csv')
  @columns = []
  @footers = {}
  return self if input.nil?
  case input
  when IO, StringIO
    case ext
    when /csv/i
      from_csv(input)
    when /org/i
      from_org(input)
    else
      raise "Don't know how to read a '#{ext}' file."
    end
  when String
    ext = File.extname(input).downcase
    File.open(input, 'r') do |io|
      case ext
      when '.csv'
        from_csv(io)
      when '.org'
        from_org(io)
      else
        raise "Don't know how to read a '#{ext}' file."
      end
    end
  when Array
    case input[0]
    when Array
      from_array_of_arrays(input)
    when Hash
      from_array_of_hashes(input)
    when Table
      from_table(input)
    else
      if input[0].respond_to?(:to_hash)
        from_array_of_hashes(input)
      else
        raise ArgumentError,
              "Cannot initialize Table with an array of #{input[0].class}"
      end
    end
  else
    raise ArgumentError,
          "Cannot initialize Table with #{input.class}"
  end
end

Instance Attribute Details

#columns ⇒ Object (readonly)

Returns the value of attribute columns.


42
43
44
 
# File 'lib/fat_core/table.rb', line 42

def columns
  @columns
end

#footers ⇒ Object (readonly)

Returns the value of attribute footers.


42
43
44
 
# File 'lib/fat_core/table.rb', line 42

def footers
  @footers
end

Instance Method Details

#<<(row) ⇒ Object 


400
401
402
 
# File 'lib/fat_core/table.rb', line 400

def <<(row)
  add_row(row)
end

#[](key) ⇒ Object

Return the array of items of the column with the given header.


100
101
102
 
# File 'lib/fat_core/table.rb', line 100

def [](key)
  column(key)
end

#add_avg_footer(cols, label = 'Average') ⇒ Object 


340
341
342
 
# File 'lib/fat_core/table.rb', line 340

def add_avg_footer(cols, label = 'Average')
  add_footer(label: label, aggregate: :avg, heads: cols)
end

#add_column(col) ⇒ Object 


404
405
406
407
408
 
# File 'lib/fat_core/table.rb', line 404

def add_column(col)
  raise "Table already has a column with header '#{col.header}'" if column?(col.header)
  columns << col
  self
end

#add_footer(label: 'Total', aggregate: :sum, heads: []) ⇒ Object 


326
327
328
329
330
331
332
333
334
 
# File 'lib/fat_core/table.rb', line 326

def add_footer(label: 'Total', aggregate: :sum, heads: [])
  foot = {}
  heads.each do |h|
    raise "No #{h} column in table to #{aggregate}" unless headers.include?(h)
    foot[h] = column(h).send(aggregate)
  end
  @footers[label.as_sym] = foot
  self
end

#add_max_footer(cols, label = 'Maximum') ⇒ Object 


348
349
350
 
# File 'lib/fat_core/table.rb', line 348

def add_max_footer(cols, label = 'Maximum')
  add_footer(label: label, aggregate: :max, heads: cols)
end

#add_min_footer(cols, label = 'Minimum') ⇒ Object 


344
345
346
 
# File 'lib/fat_core/table.rb', line 344

def add_min_footer(cols, label = 'Minimum')
  add_footer(label: label, aggregate: :min, heads: cols)
end

#add_row(row) ⇒ Object

Add a row represented by a Hash having the headers as keys. All tables should be built ultimately using this method as a primitive.


391
392
393
394
395
396
397
398
 
# File 'lib/fat_core/table.rb', line 391

def add_row(row)
  row.each_pair do |k, v|
    key = k.as_sym
    columns << Column.new(header: k) unless column?(k)
    column(key) << v
  end
  self
end

#add_sum_footer(cols, label = 'Total') ⇒ Object 


336
337
338
 
# File 'lib/fat_core/table.rb', line 336

def add_sum_footer(cols, label = 'Total')
  add_footer(heads: cols)
end

#column(key) ⇒ Object

Return the column with the given header.


95
96
97
 
# File 'lib/fat_core/table.rb', line 95

def column(key)
  columns.detect { |c| c.header == key.as_sym }
end

#column?(key) ⇒ Boolean

Returns:

  • (Boolean) 

104
105
106
 
# File 'lib/fat_core/table.rb', line 104

def column?(key)
  headers.include?(key.as_sym)
end

#empty? ⇒ Boolean

Returns:

  • (Boolean) 

133
134
135
 
# File 'lib/fat_core/table.rb', line 133

def empty?
  rows.empty?
end

#group_by(*exprs) ⇒ Object

Return a Table in which all rows of the table are divided into groups where the value of all columns named as simple symbols are equal. All other columns are set to the result of aggregating the values of that column within the group according to the Column aggregate function (:sum, :min, :max, etc.) set in a hash parameter with the non-aggregate column name as a key and the symbol for the aggregate function as a value. For example, consider the following call:

#+BEGIN_EXAMPLE tab.group_by(:date, :code, :price, shares: :sum, ). #+END_EXAMPLE

The first three parameters are simple symbols, so the table is divided into groups of rows in which the value of :date, :code, and :price are equal. The :shares parameter is set to the aggregate function :sum, so it will appear in the result as the sum of all the :shares values in each group. Any non-aggregate columns that have no aggregate function set default to using the aggregate function :first. Note that because of the way Ruby parses parameters to a method call, all the grouping symbols must appear first in the parameter list.


270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
 
# File 'lib/fat_core/table.rb', line 270

def group_by(*exprs)
  group_cols = []
  agg_cols = {}
  exprs.each do |xp|
    case xp
    when Symbol
      group_cols << xp
    when Hash
      agg_cols = xp
    else
      raise "Cannot group by parameter '#{xp}"
    end
  end
  default_agg_func = :first
  default_cols = headers - group_cols - agg_cols.keys
  default_cols.each do |h|
    agg_cols[h] = default_agg_func
  end

  sorted_tab = order_by(group_cols)
  groups = sorted_tab.rows.group_by do |r|
    group_cols.map { |k| r[k] }
  end
  result_rows = []
  groups.each_pair do |_vals, grp_rows|
    result_rows << row_from_group(grp_rows, group_cols, agg_cols)
  end
  result = Table.new
  result_rows.each do |row|
    result.add_row(row)
  end
  result
end

#headers ⇒ Object

Return the headers for the table as an array of symbols.


114
115
116
 
# File 'lib/fat_core/table.rb', line 114

def headers
  columns.map(&:header)
end

#order_by(*sort_heads) ⇒ Object

Return a new Table sorted on the rows of this Table on the possibly multiple keys given in the array of syms in headers. Append a ! to the symbol name to indicate reverse sorting on that column.


145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
 
# File 'lib/fat_core/table.rb', line 145

def order_by(*sort_heads)
  sort_heads = [sort_heads].flatten
  rev_heads = sort_heads.select { |h| h.to_s.ends_with?('!') }
  sort_heads = sort_heads.map { |h| h.to_s.sub(/\!\z/, '').to_sym }
  rev_heads = rev_heads.map { |h| h.to_s.sub(/\!\z/, '').to_sym }
  new_rows = rows.sort do |r1, r2|
    key1 = sort_heads.map { |h| rev_heads.include?(h) ? r2[h] : r1[h] }
    key2 = sort_heads.map { |h| rev_heads.include?(h) ? r1[h] : r2[h] }
    key1 <=> key2
  end
  new_tab = Table.new
  new_rows.each do |nrow|
    new_tab.add_row(nrow)
  end
  new_tab
end

#rows ⇒ Object

Return the rows of the table as an array of hashes, keyed by the headers.


119
120
121
122
123
124
125
126
127
128
129
130
131
 
# File 'lib/fat_core/table.rb', line 119

def rows
  rows = []
  unless columns.empty?
    0.upto(columns.first.items.last_i) do |rnum|
      row = {}
      columns.each do |col|
        row[col.header] = col[rnum]
      end
      rows << row
    end
  end
  rows
end

#select(*exps) ⇒ Object

Return a Table having the selected column expression. Each expression can be either a (1) symbol, (2) a hash of symbol => symbol, or (3) a hash of symbol => ‘string’, though the bare symbol arguments (1) must precede any hash arguments. Each expression results in a column in the resulting Table in the order given. The expressions are evaluated in order as well.


167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/fat_core/table.rb', line 167

def select(*exps)
  new_cols = {}
  new_heads = []
  exps.each do |exp|
    case exp
    when Symbol, String
      h = exp.as_sym
      raise "Column '#{h}' in select does not exist" unless column?(h)
      new_heads << h
      new_cols[h] = Column.new(header: h,
                               items: column(h).items)
    when Hash
      exp.each_pair do |key, xp|
        case xp
        when Symbol
          h = xp.as_sym
          raise "Column '#{key}' in select does not exist" unless column?(key)
          new_heads << h
          new_cols[h] = Column.new(header: h,
                                   items: column(key).items)
        when String
          # Evaluate xp in the context of a binding including a local
          # variable for each original column with the name as the head and
          # the value for the current row as the value and a local variable
          # for each new column with the new name and the new value.
          h = key.as_sym
          new_heads << h
          new_cols[h] = Column.new(header: h)
          ev = Evaluator.new(vars: { row: 0 }, before: '@row += 1')
          rows.each_with_index do |old_row, row_num|
            new_row ||= {}
            # Gather the new values computed so far for this row
            new_vars = new_heads.zip(new_cols.keys
                                       .map { |k| new_cols[k] }
                                       .map { |c| c[row_num] })
            vars = old_row.merge(Hash[new_vars])
            # Now we have a hash, vars, of all local variables we want to be
            # defined while evaluating expression xp as the value of column
            # key in the new column.
            new_row[h] = ev.evaluate(xp, vars: vars)
            new_cols[h] << new_row[h]
          end
        else
          raise 'Hash parameters to select must be a symbol or string'
        end
      end
    else
      raise 'Parameters to select must be a symbol, string, or hash'
    end
  end
  result = Table.new
  new_heads.each do |h|
    result.add_column(new_cols[h])
  end
  result
end

#to_org(formats: {}) ⇒ Object

This returns the table as an Array of Arrays with formatting applied. This would normally called after all calculations on the table are done and you want to return the results. The Array of Arrays structure is what org-mode src blocks will render as an org table in the buffer.


356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
 
# File 'lib/fat_core/table.rb', line 356

def to_org(formats: {})
  result = []
  header_row = []
  headers.each do |hdr|
    header_row << hdr.entitle
  end
  result << header_row
  # This causes org to place an hline under the header row
  result << nil unless header_row.empty?

  rows.each do |row|
    out_row = []
    headers.each do |hdr|
      out_row << row[hdr].format_by(formats[hdr])
    end
    result << out_row
  end
  footers.each_pair do |label, footer|
    foot_row = []
    columns.each do |col|
      hdr = col.header
      foot_row << footer[hdr].format_by(formats[hdr])
    end
    foot_row[0] = label.entitle
    result << foot_row
  end
  result
end

#types ⇒ Object

Attr_reader as a plural


109
110
111
 
# File 'lib/fat_core/table.rb', line 109

def types
  columns.map(&:type)
end

#union(other) ⇒ Object

Return a Table that combines this table with another table. The headers of this table are used in the result. There must be the same number of columns of the same type in the two tables, or an exception will be thrown. Unlike in SQL, no duplicates are eliminated from the result.


239
240
241
242
243
244
245
246
247
248
 
# File 'lib/fat_core/table.rb', line 239

def union(other)
  unless columns.size == other.columns.size
    raise 'Cannot apply union to tables with a different number of columns.'
  end
  result = Table.new
  columns.each_with_index do |col, k|
    result.add_column(col + other.columns[k])
  end
  result
end

#where(expr) ⇒ Object

Return a Table containing only rows matching the where expression.


225
226
227
228
229
230
231
232
233
 
# File 'lib/fat_core/table.rb', line 225

def where(expr)
  expr = expr.to_s
  result = Table.new
  ev = Evaluator.new(vars: { row: 0 }, before: '@row += 1')
  rows.each do |row|
    result.add_row(row) if ev.evaluate(expr, vars: row)
  end
  result
end