Class: NcsNavigator::Warehouse::Transformers::ForeignKeyIndex

Inherits:
Object
  • Object
show all
Defined in:
lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb,
lib/ncs_navigator/warehouse/transformers/foreign_key_index/static_key_provider.rb,
lib/ncs_navigator/warehouse/transformers/foreign_key_index/database_key_provider.rb

Overview

Provides for in-memory reporting of foreign key violations during ETL runs.

When FK constraint violations are delegated to the database, only one can be reported per ETL execution. This class (or one like it) allows for all foreign key violations to be reported across an entire run.

See Also:

Defined Under Namespace

Classes: DatabaseKeyProvider, StaticKeyProvider

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ ForeignKeyIndex

Returns a new instance of ForeignKeyIndex.

Parameters:

  • options (Hash<Symbol, Object>) (defaults to: {})

Options Hash (options):

  • :existing_key_provider (#existing_keys) — default: an instance of DatabaseKeyProvider —

    See #existing_key_provider for more information.



37
38
39
40
41
42
43
44
45
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 37

def initialize(options={})
  @existing_key_provider = if options.has_key?(:existing_key_provider)
    options[:existing_key_provider] || StaticKeyProvider.new # empty if nil
  else
    DatabaseKeyProvider.new
  end
  @seen_keys = {}
  @current_transform_tracker = nil
end

Instance Attribute Details

#existing_key_provider ⇒ #existing_keys (readonly)

The object that will be used to pre-initialize the known keys list for a particular model. It has a single method (existing_keys) which takes a single argument, the model class for which keys should be provided, and returns an array of the known keys for that model.

existing_keys will be called at most once for each model class. It will be called the first time either a record of the model's type is encountered or the first time a foreign key of the model's type is encountered.

Returns:

  • (#existing_keys) 


31
32
33
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 31

def existing_key_provider
  @existing_key_provider
end

Instance Method Details

#end_transform ⇒ Array<DataMapper::Resource>

Reviews any references that initially failed against the final set of keys. If any are still unresolvable, it records errors against the provided transform status.

Each failed reference will be reported by this method only once. This is so that it is safe to share a single instance of this class across multiple transformers, reporting only the newly encountered errors for each transform in turn.

Returns:

  • (Array<DataMapper::Resource>) —

    any records which were previously deferred but which now are fully resolved and are candidates to be persisted.



104
105
106
107
108
109
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 104

def end_transform
  fail "No current transform" unless @current_transform_tracker
  @current_transform_tracker.end_it(self).tap do |x|
    @current_transform_tracker =  nil
  end
end

#record(record)

This method returns an undefined value.

Records the key for this record in the index. By calling this method, the caller affirms that the record should be considered available for resolution of future foreign keys.

Parameters:

  • record (DataMapper::Resource) —

    the record whose key we want to record.



87
88
89
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 87

def record(record)
  seen_keys(record.class) << record.key.first # no CPKs in MDES
end

#seen?(model_class, id) ⇒ Boolean

Returns has the index seen a record of the given type with the given ID.

Returns:

  • (Boolean) —

    has the index seen a record of the given type with the given ID.



114
115
116
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 114

def seen?(model_class, id)
  seen_keys(model_class).include?(id)
end

#start_transform(transform_status) ⇒ Object

Indicates the beginning of a new transform. This method must be called before the first time #verify_or_defer is called for a particular transform.

Error reporting and deferred foreign key resolution are scoped to a transform.



53
54
55
56
57
58
59
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 53

def start_transform(transform_status)
  if @current_transform_tracker
    fail "#start_transform called before previous transform's #end_transform called. This will lose deferred records."
  else
    @current_transform_tracker = TransformTracker.new(transform_status)
  end
end

#verify_or_defer(record) ⇒ Boolean

Indicates whether a record's foreign keys can immediately be satisfied. If not, it stores the record for processing in and possible return from #end_transform.

Parameters:

  • record (DataMapper::Resource) —

    the record whose foreign references we want to verify.

Returns:

  • (Boolean) 


69
70
71
72
73
74
75
76
77
 
# File 'lib/ncs_navigator/warehouse/transformers/foreign_key_index.rb', line 69

def verify_or_defer(record)
  deferrer = DeferredRecord.create_if_appropriate(record, self)
  if deferrer
    @current_transform_tracker.defer(deferrer)
    false
  else
    true
  end
end