Class: NdrImport::Table
- Inherits:
-
Object
- Object
- NdrImport::Table
- Includes:
- Mapper
- Defined in:
- lib/ndr_import/table.rb
Overview
This class maintains the state of a table mapping and encapsulates the logic required to transform a table of data into “records”. Particular attention has been made to use enumerables throughout to help with the transformation of large quantities of data. rubocop:disable Metrics/ClassLength
Direct Known Subclasses
Avro::Table, FixedWidth::Table, NonTabular::Table, PdfForm::Table, Vcf::Table, Xml::Table
Instance Attribute Summary collapse
-
#notifier ⇒ Object
Returns the value of attribute notifier.
Class Method Summary collapse
Instance Method Summary collapse
- #all_valid_options ⇒ Object
-
#encode_with(coder) ⇒ Object
For readability, we should serialise the columns last.
- #header_valid? ⇒ Boolean
-
#initialize(options = {}) ⇒ Table
constructor
A new instance of Table.
- #match(filename, tablename) ⇒ Object
-
#process_line(line, &block) ⇒ Object
This method process a line of data, If it is a header line it validates it, otherwise transforms it.
-
#transform(lines, &block) ⇒ Object
This method transforms a table of data, given a line array/enumerator and yields klass, fields and index (input row number) for each record that it would create as a result of the transformation process.
-
#transform_line(line, index) ⇒ Object
This method transforms an incoming line of data by applying each of the klass masked mappings to the line and yielding the klass and fields for each mapped klass.
Constructor Details
#initialize(options = {}) ⇒ Table
Returns a new instance of Table.
26 27 28 29 30 31 32 33 34 35 36 37 |
# File 'lib/ndr_import/table.rb', line 26 def initialize( = {}) .stringify_keys! if .is_a?(Hash) () .each do |key| # This pattern is used to only set attributes if option specified, # which makes for more concise YAML serialization. [key] && instance_variable_set("@#{key}", [key]) end @row_index = 0 end |
Instance Attribute Details
#notifier ⇒ Object
Returns the value of attribute notifier.
24 25 26 |
# File 'lib/ndr_import/table.rb', line 24 def notifier @notifier end |
Class Method Details
.all_valid_options ⇒ Object
13 14 15 16 17 |
# File 'lib/ndr_import/table.rb', line 13 def self. %w[canonical_name delimiter liberal_parsing filename_pattern file_password last_data_column tablename_pattern header_lines footer_lines format klass columns slurp row_identifier significant_mapped_fields] end |
Instance Method Details
#all_valid_options ⇒ Object
19 20 21 |
# File 'lib/ndr_import/table.rb', line 19 def self.class. end |
#encode_with(coder) ⇒ Object
For readability, we should serialise the columns last
107 108 109 110 111 112 113 114 |
# File 'lib/ndr_import/table.rb', line 107 def encode_with(coder) = self.class. - ['columns'] .each do |option| value = send(option) coder[option] = value if value end coder['columns'] = @columns end |
#header_valid? ⇒ Boolean
102 103 104 |
# File 'lib/ndr_import/table.rb', line 102 def header_valid? @header_valid == true end |
#match(filename, tablename) ⇒ Object
39 40 41 42 |
# File 'lib/ndr_import/table.rb', line 39 def match(filename, tablename) ::File.basename(filename) =~ (filename_pattern || /\A.*\z/) && (tablename.nil? || tablename =~ (tablename_pattern || /\A.*\z/)) end |
#process_line(line, &block) ⇒ Object
This method process a line of data, If it is a header line it validates it, otherwise transforms it. It also increments the row index and notifies the amount of lines processed.
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 |
# File 'lib/ndr_import/table.rb', line 65 def process_line(line, &block) return enum_for(:process_line, line) unless block if @row_index < header_lines consume_header_line(line, @columns) else transform_line(line, @row_index, &block) end @row_index += 1 # We've now seen enough lines to have consumed a valid header; is this the case? fail_unless_header_complete(@columns) if @row_index == header_lines @notifier.try(:processed, @row_index) end |
#transform(lines, &block) ⇒ Object
This method transforms a table of data, given a line array/enumerator and yields klass, fields and index (input row number) for each record that it would create as a result of the transformation process.
47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
# File 'lib/ndr_import/table.rb', line 47 def transform(lines, &block) return enum_for(:transform, lines) unless block @row_index = 0 @header_valid = false @header_best_guess = nil @notifier.try(:started) last_col = last_column_to_transform (lines, ).each do |line| line.is_a?(Array) ? process_line(line[0..last_col], &block) : process_line(line, &block) end @notifier.try(:finished) end |
#transform_line(line, index) ⇒ Object
This method transforms an incoming line of data by applying each of the klass masked mappings to the line and yielding the klass and fields for each mapped klass.
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
# File 'lib/ndr_import/table.rb', line 84 def transform_line(line, index) return enum_for(:transform_line, line, index) unless block_given? identifier = case @row_identifier.to_s when 'index' index when 'uuid' SecureRandom.uuid end masked_mappings.each do |klass, klass_mappings| fields = mapped_line(line, klass_mappings) fields['row_identifier'] = identifier unless identifier.nil? next if fields[:skip].to_s == 'true' yield(klass, fields, index) end end |