Class: NcsNavigator::Warehouse::Configuration

Inherits:
Object
  • Object
show all
Defined in:
lib/ncs_navigator/warehouse/configuration.rb,
lib/ncs_navigator/warehouse/configuration/file_evaluator.rb

Overview

The configuration profile for the warehouse in a particular environment. An instance of this class is derived from the environment script under /etc/nubic/ncs/warehouse/{env_name}.rb.

See Also:

Defined Under Namespace

Classes: Error

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#bcdatabase_group ⇒ Symbol

Returns the bcdatabase group to use when locating the database configuration for this instance. It will also be used by default in Transformers::Database transformers.

Returns:

  • (Symbol) —

    the bcdatabase group to use when locating the database configuration for this instance. It will also be used by default in Transformers::Database transformers.



355
356
357
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 355

def bcdatabase_group
  @bcdatabase_group || default_bcdatabase_group
end

#configuration_file ⇒ Pathname?

The file from which this configuration was originally read. If the configuration was constructed solely in memory, this value will be nil.

Returns:

  • (Pathname, nil) 


538
539
540
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 538

def configuration_file
  @configuration_file
end

#default_xml_filter_set ⇒ Symbol?

Returns the filter set to with the XML emitter if none is specified.

Returns:

  • (Symbol, nil) —

    the filter set to with the XML emitter if none is specified.



201
202
203
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 201

def default_xml_filter_set
  @default_xml_filter_set
end

#mdes ⇒ NcsNavigator::Mdes::Specification

Returns the specification for the active MDES version.

N.b.: this method triggers loading the default MDES module and specification if #mdes_version= has not been called yet.

Returns:

  • (NcsNavigator::Mdes::Specification) —

    the specification (provided by ncs_mdes) for the active MDES version.



260
261
262
263
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 260

def mdes
  set_default_mdes_version unless @mdes
  @mdes
end

#models_module ⇒ Module

Returns the module namespacing the models for the active MDES version.

N.b.: this method triggers loading the default MDES module and specification if #mdes_version= has not been called yet.

Returns:

  • (Module) —

    the module namespacing the models for the active MDES version.



274
275
276
277
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 274

def models_module
  set_default_mdes_version unless @models_module
  @models_module
end

#navigator ⇒ NcsNavigator::Configuration

Returns a suite configuration object. Defaults to the global default instance (NcsNavigator.configuration) which is loaded from /etc/nubic/ncs/navigator.ini.

Returns:

  • (NcsNavigator::Configuration) —

    a suite configuration object. Defaults to the global default instance (NcsNavigator.configuration) which is loaded from /etc/nubic/ncs/navigator.ini.



310
311
312
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 310

def navigator
  @navigator ||= NcsNavigator.configuration
end

#pg_bin_path ⇒ Pathname?

The path where the PostgreSQL command line utilities can be found. If they are on the search path, this may be nil (the default).

Returns:

  • (Pathname, nil) 


504
505
506
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 504

def pg_bin_path
  @pg_bin_path
end

#shell_io ⇒ IO

The IO to use for terminal monitoring output. Defaults to standard error. Use #shell to actually write to it.

Returns:

  • (IO) 


428
429
430
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 428

def shell_io
  @shell_io ||= $stderr
end

Class Method Details

.environment_file(env_name = nil) ⇒ Object 



52
53
54
55
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 52

def environment_file(env_name=nil)
  env_name ||= NcsNavigator::Warehouse.env
  "/etc/nubic/ncs/warehouse/#{env_name}.rb"
end

.for_environment(env_name = nil) ⇒ Configuration

Loads the configuration for the named environment from /etc/nubic/ncs/warehouse/{env_name}.rb.

Parameters:

  • env_name (#to_s) (defaults to: nil)

Returns:

  • (Configuration) —

    the configuration loaded from the configuration file for the named environment.



43
44
45
46
47
48
49
50
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 43

def for_environment(env_name=nil)
  fn = environment_file env_name
  if File.exist?(fn)
    from_file(fn)
  else
    new
  end
end

.from_file(filename) ⇒ Configuration

Evaluates the given file to initialize a NcsNavigator::Warehouse::Configuration object. See sample_configuration for an example.

Parameters:

  • filename (String)

Returns:

  • (Configuration) —

    a configuration initialized from the given file.



32
33
34
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 32

def from_file(filename)
  FileEvaluator.new(filename.to_s).result
end

Instance Method Details

#add_filter_set(name, one_or_more_filters)

This method returns an undefined value.

Define a named filter set.

Parameters:

  • name (Symbol, #to_sym) —

    the name for this set.

  • one_or_more_filters (#call, Array<#call>) —

    the filters to use in the set



171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 171

def add_filter_set(name, one_or_more_filters)
  if filter_sets.has_key?(name.to_sym)
    raise Error, "There is already a filter set named #{name.inspect}."
  end

  filters = [*one_or_more_filters].collect do |candidate|
    case candidate
    when Symbol
      filter_set(candidate)
    else
      candidate
    end
  end

  filters.each do |candidate|
    unless candidate.respond_to?(:call)
      if candidate.respond_to?(:new)
        raise Error, "#{candidate.inspect} does not have a call method. Perhaps you meant #{candidate.inspect}.new?"
      else
        raise Error, "#{candidate.inspect} does not have a call method."
      end
    end
  end

  filter_sets[name.to_sym] = Filters::CompositeFilter.new(filters)
end

#add_post_etl_hook(candidate)

This method returns an undefined value.

Adds a post-ETL hook to the list for this warehouse instance.

Parameters:

  • the (#etl_succeeded, #etl_failed) —

    hook



122
123
124
125
126
127
128
129
130
131
132
133
134
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 122

def add_post_etl_hook(candidate)
  expected_methods = [:etl_succeeded, :etl_failed]
  implemented_methods = expected_methods.select { |m| candidate.respond_to?(m) }
  if implemented_methods.empty?
    msg = "#{candidate.inspect} does not have an #{expected_methods.join(' or ')} method."
    if candidate.respond_to?(:new)
      msg += " Perhaps you meant #{candidate}.new?"
    end
    raise Error, msg
  else
    post_etl_hooks << candidate
  end
end

#add_transformer(candidate)

This method returns an undefined value.

Adds a transformer to the list for this warehouse instance.

Parameters:

  • the (#transform) —

    transformer object.



73
74
75
76
77
78
79
80
81
82
83
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 73

def add_transformer(candidate)
  if candidate.respond_to?(:transform)
    self.transformers << candidate
  else
    if candidate.respond_to?(:new)
      raise Error, "#{candidate.inspect} does not have a transform method. Perhaps you meant #{candidate.inspect}.new?"
    else
      raise Error, "#{candidate.inspect} does not have a transform method."
    end
  end
end

#bcdatabase_entries ⇒ Hash<Symbol, Symbol>

Returns the bcdatabase entries to use when locating the database configurations for this instance. It must have the keys :working and :reporting.

Returns:

  • (Hash<Symbol, Symbol>) —

    the bcdatabase entries to use when locating the database configurations for this instance. It must have the keys :working and :reporting.



386
387
388
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 386

def bcdatabase_entries
  @bcdatabase_entries ||= default_bcdatabase_entries
end

#filter_set(name) ⇒ #call

Looks up a filter set by name. Errors out if no match found.

If you need to refer to a named filter set in your configuration file, use this method instead of #filter_sets. E.g.:

c.add_transformer SomeDatabase.create_transformer(c, :filters => [c.filter_set(:some_filters)])

Using this method instead of #filter_sets ensures that you will get a useful error message if you have a typo in your filter name.

Returns:

  • (#call) —

    the CompositeFilter registered under the given name. If none, it raises Error.



160
161
162
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 160

def filter_set(name)
  filter_sets[name.to_sym] or raise Error, "Unknown filter set #{name.inspect}."
end

#filter_sets ⇒ Hash<Symbol,#call>

Returns an index of named CompositeFilters reflecting the configured named filter sets.

Returns:

  • (Hash<Symbol,#call>) —

    an index of named CompositeFilters reflecting the configured named filter sets



143
144
145
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 143

def filter_sets
  @filter_sets ||= {}
end

#foreign_key_index ⇒ Object

The foreign key index used during an ETL run. The default value is correct for virtually any case.

Returns:

  • (Object) 


90
91
92
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 90

def foreign_key_index
  @foreign_key_index ||= Transformers::ForeignKeyIndex.new
end

#foreign_key_index=(index)

This method returns an undefined value.

Specify a different foreign key index implementation to use. This will only rarely be necessary or useful. The default value is correct for virtually any case.

Parameters:

  • index (#start_transform, #verify_or_defer, #record, #end_transform) —

    the replacement foreign key index implementation.



102
103
104
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 102

def foreign_key_index=(index)
  @foreign_key_index = index
end

#log ⇒ Logger

Returns the primary application log.

Returns:

  • (Logger) —

    the primary application log



489
490
491
492
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 489

def log
  set_up_logs unless @log
  @log
end

#log_file ⇒ Pathname

The file to which the warehouse will log its actions. Defaults to /var/log/ncs/warehouse/{env_name}.log.

Returns:

  • (Pathname) 


458
459
460
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 458

def log_file
  @log_directory ||= Pathname.new("/var/log/nubic/ncs/warehouse/#{env}.log")
end

#log_file=(fn) ⇒ Object

Specify the filename for the warehouse's logs.

Parameters:

  • fn (Pathname, String, nil) 


466
467
468
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 466

def log_file=(fn)
  @log_directory = coerce_to_pathname(fn)
end

#mdes_version ⇒ String

Returns the configured MDES version string.

N.b.: this method triggers loading the default MDES module and specification if #mdes_version= has not been called yet.

Returns:

  • (String) —

    the configured MDES version.



248
249
250
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 248

def mdes_version
  mdes.version
end

#mdes_version=(version_number)

This method returns an undefined value.

Selects and loads a set of models based on the given warehouse version. Also initializes #mdes and #models_module.

Parameters:

  • version_number (String) —

    the version of the MDES to use.



226
227
228
229
230
231
232
233
234
235
236
237
238
239
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 226

def mdes_version=(version_number)
  module_name = TableModeler.version_module_name(version_number)
  module_require = "ncs_navigator/warehouse/models/#{module_name.underscore}"

  begin
    require module_require
  rescue LoadError => e
    raise Error, "No warehouse models exist for MDES version #{version_number}: #{e}"
  end

  self.mdes = NcsNavigator::Mdes(version_number)

  @models_module = NcsNavigator::Warehouse::Models.const_get module_name
end

#model(table_or_model_name) ⇒ Class?

Returns the model in the current MDES version corresponding to the given model or table name, or nil if there is no such model.

Parameters:

  • table_or_model_name (String, Symbol) —

    either an MDES table name or the unqualified name of an MDES Warehouse model

Returns:

  • (Class, nil) —

    the model in the current MDES version corresponding to the given model or table name, or nil if there is no such model



292
293
294
295
296
297
298
299
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 292

def model(table_or_model_name)
  begin
    models_module.const_get(table_or_model_name)
  rescue NameError
    models_module.mdes_order.
      find { |model| model.mdes_table_name.to_s == table_or_model_name.to_s }
  end
end

#navigator_ini=(ini_file)

This method returns an undefined value.

Set the suite configuration file to use in this warehouse environment.

Parameters:

  • ini_file (String) —

    the name of an INI file that is compatible with NcsNavigator::Configuration.

See Also:

  • NcsNavigator::Warehouse::Configuration.{{#navigator}


323
324
325
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 323

def navigator_ini=(ini_file)
  @navigator = NcsNavigator::Configuration.new(ini_file)
end

#output_level ⇒ :normal, :quiet

Returns the desired terminal output level. This does not affect logging. Default is :normal.

Returns:

  • (:normal, :quiet) —

    the desired terminal output level. This does not affect logging. Default is :normal.



405
406
407
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 405

def output_level
  @output_level ||= :normal
end

#output_level=(level)

This method returns an undefined value.

Set the desired terminal output level.

Parameters:

  • level (:normal, :quiet) 


414
415
416
417
418
419
420
421
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 414

def output_level=(level)
  level = level.try(:to_sym)
  if [:normal, :quiet].include?(level)
    @output_level = level
  else
    fail Error, "#{level.inspect} is not a valid value for output_level."
  end
end

#pg_bin(command) ⇒ Pathname

Returns the executable for the given PostgreSQL utility.

Parameters:

  • command (Pathname, String) —

    the name of the command

Returns:

  • (Pathname) —

    the executable for the given PostgreSQL utility.



520
521
522
523
524
525
526
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 520

def pg_bin(command)
  if pg_bin_path
    pg_bin_path + command
  else
    coerce_to_pathname command
  end
end

#post_etl_hooks ⇒ Array<#etl_succeeded,#etl_failed>

Returns the configured post-ETL hooks.

Returns:

  • (Array<#etl_succeeded,#etl_failed>) —

    the configured post-ETL hooks.



113
114
115
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 113

def post_etl_hooks
  @post_etl_hooks ||= []
end

#set_up_action_mailer

This method returns an undefined value.

Configures ActionMailer with the options implied by the suite configuration.



336
337
338
339
340
341
342
343
344
345
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 336

def set_up_action_mailer
  return if @action_mailer_set_up
  require 'action_mailer'
  ActionMailer::Base.delivery_method = :smtp
  ActionMailer::Base.smtp_settings = navigator.action_mailer_smtp_settings
  ActionMailer::Base.view_paths = [
    File.expand_path('../mailer_templates', __FILE__)
  ]
  @action_mailer_set_up = true
end

#set_up_logs ⇒ Object 



470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 470

def set_up_logs
  if log_file.writable? || log_file.parent.writable?
    @log = Logger.new(log_file).tap do |l|
      l.level = Logger::DEBUG
    end
    ::DataMapper::Logger.new(log_file, :debug, " DM: ", true)
  else
    @log = Logger.new($stdout).tap do |l|
      l.level = Logger::WARN
    end
    ::DataMapper::Logger.new($stdout, :warn, " DM: ", true)

    $stdout.puts "WARNING: Could not create or update log #{log_file}."
    $stdout.puts 'WARNING: Will log errors and warnings to standard out until this is fixed.'
  end
end

#shell ⇒ Object

The terminal output shell for command line components to use. It will be an UpdatingShell or something which behaves like one.



437
438
439
440
441
442
443
444
445
446
447
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 437

def shell
  @shell ||=
    case output_level
    when :normal
      UpdatingShell.new(shell_io)
    when :quiet
      UpdatingShell::Quiet.new
    else
      fail "Unexpected output_level #{output_level.inspect}"
    end
end

#transformers ⇒ Array<#transform>

Returns the configured transformer objects.

Returns:

  • (Array<#transform>) —

    the configured transformer objects.



64
65
66
 
# File 'lib/ncs_navigator/warehouse/configuration.rb', line 64

def transformers
  @transformers ||= []
end