Class: Fech::Filing

Inherits:
Object
  • Object
show all
Defined in:
lib/fech/filing.rb

Overview

Fech::Filing downloads an Electronic Filing given its ID, and will search rows by row type. Using a child Translator object, the data in each row is automatically mapped at runtime into a labeled Hash. Additional Translations may be added to change the way that data is mapped and cleaned.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(filing_id, opts = {}) ⇒ Filing

Create a new Filing object, assign the download directory to system’s temp folder by default.

Parameters:

  • download_dir (String) —

    override the directory where files should be downloaded.

  • translate (Symbol, Array) —

    a list of built-in translation sets to use



18
19
20
21
22
23
24
 
# File 'lib/fech/filing.rb', line 18

def initialize(filing_id, opts={})
  @filing_id    = filing_id
  @download_dir = opts[:download_dir] || Dir.tmpdir
  @translator   = Fech::Translator.new(:include => opts[:translate])
  @quote_char   = opts[:quote_char] || '"'
  @csv_parser   = opts[:csv_parser] || Fech::Csv
end

Instance Attribute Details

#download_dir ⇒ Object

Returns the value of attribute download_dir.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def download_dir
  @download_dir
end

#filing_id ⇒ Object

Returns the value of attribute filing_id.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def filing_id
  @filing_id
end

#translator ⇒ Object

Returns the value of attribute translator.



11
12
13
 
# File 'lib/fech/filing.rb', line 11

def translator
  @translator
end

Class Method Details

.map_for(row_type, opts = {}) ⇒ Object

Returns the column names for given row type and version in the order they appear in row data.

Parameters:

  • row_type (String, Regexp) —

    representation of the row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :version (String, Regexp) —

    representation of the version desired



153
154
155
 
# File 'lib/fech/filing.rb', line 153

def self.map_for(row_type, opts={})
  Fech::Mappings.for_row(row_type, opts)
end

Instance Method Details

#amendment? ⇒ Boolean

Whether this filing amends a previous filing or not.

Returns:

  • (Boolean) 


168
169
170
 
# File 'lib/fech/filing.rb', line 168

def amendment?
  !amends.nil?
end

#amends ⇒ Object

Returns the filing ID of the past filing this one amends, nil if this is a first-draft filing. :report_id in the HDR line references the amended filing



175
176
177
 
# File 'lib/fech/filing.rb', line 175

def amends
  header[:report_id]
end

#delimiter ⇒ String

Returns the delimiter used in the filing’s version.

Returns:

  • (String) —

    the delimiter used in the filing’s version



248
249
250
 
# File 'lib/fech/filing.rb', line 248

def delimiter
  filing_version.to_f < 6 ? "," : "\034"
end

#download ⇒ Object

Saves the filing data from the FEC website into the default download directory.



28
29
30
31
32
33
 
# File 'lib/fech/filing.rb', line 28

def download
  File.open(file_path, 'w') do |file|
    file << open(filing_url).read
  end
  self
end

#each_row(opts = {}) {|Array| ... } ⇒ Object

Iterates over and yields the Filing’s lines

Parameters:

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :with_index (Boolean) —

    yield both the item and its index

Yields:

  • (Array) —

    a row of the filing, split by the delimiter from #delimiter



226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
 
# File 'lib/fech/filing.rb', line 226

def each_row(opts={}, &block)
  unless File.exists?(file_path)
    raise "File #{file_path} does not exist. Try invoking the .download method on this Filing object."
  end

  c = 0
  @csv_parser.parse_row(file_path, :col_sep => delimiter, :quote_char => @quote_char, :skip_blanks => true) do |row|
    if opts[:with_index]
      yield [row, c]
      c += 1
    else
      yield row
    end
  end
end

#each_row_with_index(&block) ⇒ Object

Wrapper around .each_row to include indexes



243
244
245
 
# File 'lib/fech/filing.rb', line 243

def each_row_with_index(&block)
  each_row(:with_index => true, &block)
end

#file_name ⇒ Object 



215
216
217
 
# File 'lib/fech/filing.rb', line 215

def file_name
  "#{filing_id}.fec"
end

#file_path ⇒ Object

The location of the Filing on the file system



211
212
213
 
# File 'lib/fech/filing.rb', line 211

def file_path
  File.join(download_dir, file_name)
end

#filing_url ⇒ Object 



219
220
221
 
# File 'lib/fech/filing.rb', line 219

def filing_url
  "http://query.nictusa.com/dcdev/posted/#{filing_id}.fec"
end

#filing_version ⇒ Object

The version of the FEC software used to generate this Filing



189
190
191
 
# File 'lib/fech/filing.rb', line 189

def filing_version
  @filing_version ||= parse_filing_version
end

#hash_zip(keys, values) ⇒ Fech::Mapped, Hash

Combines an array of keys and values into an Fech::Mapped object, a type of Hash.

Parameters:

  • keys (Array) —

    the desired keys for the new hash

  • values (Array) —

    the desired values for the new hash

Returns:



184
185
186
 
# File 'lib/fech/filing.rb', line 184

def hash_zip(keys, values)
  Fech::Mapped.new(self, values.first).merge(Hash[*keys.zip(values).flatten])
end

#header(opts = {}) ⇒ Hash

Access the header (first) line of the filing, containing information about the filing’s version and metadata about the software used to file it.

Returns:

  • (Hash) —

    a hash that assigns labels to the values of the filing’s header row



38
39
40
41
42
 
# File 'lib/fech/filing.rb', line 38

def header(opts={})
  each_row do |row|
    return parse_row?(row)
  end
end

#map(row, opts = {}) ⇒ Object

Maps a raw row to a labeled hash following any rules given in the filing’s Translator based on its version and row type. Finds the correct map for a given row, performs any matching Translations on the individual values, and returns either the entire dataset, or just those fields requested.

Parameters:

  • row (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :include (Array) —

    list of field names that should be included in the returned hash



104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/fech/filing.rb', line 104

def map(row, opts={})
  data = Fech::Mapped.new(self, row.first)
  full_row_map = map_for(row.first)
  
  # If specific fields were asked for, return only those
  if opts[:include]
    row_map = full_row_map.select { |k| opts[:include].include?(k) }
  else
    row_map = full_row_map
  end
  
  # Inserts the row into data, performing any specified preprocessing
  # on individual cells along the way
  row_map.each_with_index do |field, index|
    value = row[full_row_map.index(field)]
    translator.get_translations(:row => row.first,
        :version => filing_version, :action => :convert,
        :field => field).each do |translation|
      # User's Procs should be given each field's value as context
      value = translation[:proc].call(value)
    end
    data[field] = value
  end
  
  # Performs any specified group preprocessing / combinations
  combinations = translator.get_translations(:row => row.first,
        :version => filing_version, :action => :combine)
  row_hash = hash_zip(row_map, row) if combinations
  combinations.each do |translation|
    # User's Procs should be given the entire row as context
    value = translation[:proc].call(row_hash)
    field = translation[:field].source.gsub(/[\^\$]*/, "").to_sym
    data[field] = value
  end
  
  data
end

#map_for(row_type) ⇒ Object

Returns the column names for given row type and the filing’s version in the order they appear in row data.

Parameters:

  • row_type (String, Regexp) —

    representation of the row desired



145
146
147
 
# File 'lib/fech/filing.rb', line 145

def map_for(row_type)
  mappings.for_row(row_type)
end

#mappings ⇒ Object

Gets or creats the Mappings instance for this filing_version



206
207
208
 
# File 'lib/fech/filing.rb', line 206

def mappings
  @mapping ||= Fech::Mappings.new(filing_version)
end

#parse_filing_version ⇒ Object

Pulls out the version number from the header line. Must parse this line manually, since we don’t know the version yet, and thus the delimiter type is still a mystery.



196
197
198
199
200
201
202
203
 
# File 'lib/fech/filing.rb', line 196

def parse_filing_version
  first = File.open(file_path).first
  if first.index("\034").nil?
    @csv_parser.parse(first).flatten[2]
  else
    @csv_parser.parse(first, :col_sep => "\034").flatten[2]
  end
end

#parse_row?(row, opts = {}) ⇒ Boolean

Decides what to do with a given row. If the row’s type matches the desired type, or if no type was specified, it will run the row through #map. If :raw was passed true, a flat, unmapped data array will be returned.

Parameters:

  • row (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :include (Array) —

    list of field names that should be included in the returned hash

Returns:

  • (Boolean) 


86
87
88
89
90
91
92
93
94
 
# File 'lib/fech/filing.rb', line 86

def parse_row?(row, opts={})
  # Always parse, unless :parse_if is given and does not match row
  if opts[:parse_if].nil? || \
      Fech.regexify(opts[:parse_if]).match(row.first.downcase)
    opts[:raw] ? row : map(row, opts)
  else
    false
  end
end

#rows_like(row_type, opts = {}) {|Hash| ... } ⇒ Array

Access all lines of the filing that match a given row type. Will return an Array of all available lines if called directly, or will yield the mapped rows one by one if a block is passed.

Parameters:

  • row_type (String, Regexp) —

    a partial or complete name of the type of row desired

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :raw (Boolean) —

    should the function return the data as an array that has not been mapped to column names

  • :include (Array) —

    list of field names that should be included in the returned hash

Yields:

  • (Hash) —

    each matched row’s data, as either a mapped hash or raw array

Returns:

  • (Array) —

    the complete set of mapped hashes for matched lines



65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'lib/fech/filing.rb', line 65

def rows_like(row_type, opts={}, &block)
  data = []
  each_row do |row|
    value = parse_row?(row, opts.merge(:parse_if => row_type))
    next if value == false
    if block_given?
      yield value
    else
      data << value if value
    end
  end
  block_given? ? nil : data
end

#summary ⇒ Hash

Access the summary (second) line of the filing, containing aggregate and top-level information about the filing.

Returns:

  • (Hash) —

    a hash that assigns labels to the values of the filing’s summary row



47
48
49
50
51
52
 
# File 'lib/fech/filing.rb', line 47

def summary
  each_row_with_index do |row, index|
    next if index == 0
    return parse_row?(row)
  end
end

#translate {|t| ... } ⇒ Object

Yields:

  • (t) —

    returns a reference to the filing’s Translator

Yield Parameters:



159
160
161
162
163
164
165
 
# File 'lib/fech/filing.rb', line 159

def translate(&block)
  if block_given?
    yield translator
  else
    translator
  end
end