Module: NcsNavigator::Warehouse::Transformers::Database

Extended by:
Forwardable
Includes:
Enumerable
Defined in:
lib/ncs_navigator/warehouse/transformers/database.rb

Overview

A mixin that provides a DSL for defining a series of application database queries and the logic for transforming their results into warehouse model instances. Example:

class StaffPortalTransformer
  include NcsNavigator::Warehouse::Transformers::Database

  bcdatabase :name => 'ncs_staff_portal'

  produce_records(:staff) do |row, meta|
    staff_model = meta[:configuration].model(:staff)
    staff_model.new(
      :staff_id => 'SP' + row.username
      # etc.
    )
  end

  produce_records(
    :staff_languages,
    :query => %Q(
      SELECT sub.*, s.username
      FROM staff_languages sub INNER JOIN staff s ON sub.staff_id=s.id
    )
  ) do |row, meta|
    sl_model = meta[:configuration].model(:staff_language)
    sl_model.new(
      :staff_language_id => 'SP' + row.id,
      :staff_id => 'SP' + row.username,
      :staff_lang => row.lang_code
      # etc.
    )
  end
end

This mixin creates an Enumerable that executes all the defined productions and streams their resulting values. As such, it is compatible with EnumTransformer.

See Also:

Defined Under Namespace

Modules: DSL, Factory Classes: OneForOneProducer, RecordProducer, UnusedColumnsForModelError

Instance Method Summary collapse

Instance Method Details

#bcdatabase ⇒ Hash

Returns the bcdatabase group and name to use to set up the repository for the application.

Returns:

  • (Hash) —

    the bcdatabase group and name to use to set up the repository for the application.



76
77
78
 
# File 'lib/ncs_navigator/warehouse/transformers/database.rb', line 76

def bcdatabase
  @bcdatabase ||= self.class.bcdatabase
end

#each(*producer_names)

This method returns an undefined value.

The main entry point for a class that mixes in this module. It evaluates the instructions defined by the DSL and streams the results to the provided block.

Parameters:

  • producer_names (Array<Symbol>) —

    if listed, only execute the named producers. Intended for isolated testing of each defined producer.



111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/ncs_navigator/warehouse/transformers/database.rb', line 111

def each(*producer_names)
  producers = selected_producers(producer_names)
  producer_name_length = producers.collect { |rp| rp.name.to_s.size }.max
  row_count = 0
  result_count = 0

  log.info(
    "Producing records from #{self.class} (#{producers.collect { |p| p.name }.join(', ')})")

  producers.each do |rp|
    shell.clear_line_then_say(
      "Producing records from %-#{producer_name_length}s (%-24s)" % [rp.name, 'loading'])
    log.debug("Executing query for producer #{rp.name}:\n#{rp.query}")
    repository.adapter.select(rp.query).each do |row|
      meta = { :configuration => @configuration }
      args = [row]
      args << meta if rp.row_processor.arity > 1

      row_count += 1
      [*rp.row_processor.call(*args)].compact.each do |result|
        yield result
        result_count += 1
        shell.back_up_and_say(26, "(%-7d in / %-7d out)" % [row_count, result_count])
      end
      shell.back_up_and_say(26, "(%-7d in / %-7d out)" % [row_count, result_count])
    end
    shell.back_up_and_say(26, "(%-7d in / %-7d out)" % [row_count, result_count])
    log.debug("Producer #{rp.name} complete")
  end
  shell.clear_line_then_say("#{self.class} complete (%-7d in / %-7d out)\n" % [row_count, result_count])

  log.info(
    "Production from #{self.class} complete. " +
    "#{result_count} MDES record#{'s' if result_count != 1} created from " +
    "#{row_count} row#{'s' if row_count != 1}.")

  nil
end

#initialize(configuration, options = {}) ⇒ Object 



59
60
61
62
63
64
 
# File 'lib/ncs_navigator/warehouse/transformers/database.rb', line 59

def initialize(configuration, options={})
  @configuration = configuration
  @repository_name = options.delete(:repository) || options.delete(:repository_name)
  @bcdatabase = { :group => configuration.bcdatabase_group }.
    merge((self.class.bcdatabase.merge(options.delete(:bcdatabase) || {})))
end

#repository ⇒ DataMapper::Repository

Returns the repository to use for querying application data.

Returns:

  • (DataMapper::Repository) —

    the repository to use for querying application data.



92
93
94
95
96
97
98
99
 
# File 'lib/ncs_navigator/warehouse/transformers/database.rb', line 92

def repository
  @repository ||=
    begin
      log.debug("Connecting DM repo #{repository_name.inspect} with #{connection_parameters.merge('password' => 'SUPPRESSED').inspect}")
      ::DataMapper.setup(repository_name, connection_parameters)
      ::DataMapper.repository(repository_name)
    end
end

#repository_name ⇒ Symbol

Returns the name of the repository to use / set up for this instance.

Returns:

  • (Symbol) —

    the name of the repository to use / set up for this instance.



69
70
71
 
# File 'lib/ncs_navigator/warehouse/transformers/database.rb', line 69

def repository_name
  @repository_name ||= self.class.repository_name
end