Class: InstDataShipper::Destinations::Base

Inherits:
Object
  • Object
show all
Defined in:
lib/inst_data_shipper/destinations/base.rb

Direct Known Subclasses

HostedData, S3, Speccable

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(cache_key, config, dumper) ⇒ Base

Returns a new instance of Base.



8
9
10
11
12
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 8

def initialize(cache_key, config, dumper)
  @cache_key = cache_key
  @_config = config
  @dumper = dumper
end

Instance Attribute Details

#dumper ⇒ Object (readonly)

Returns the value of attribute dumper.



4
5
6
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 4

def dumper
  @dumper
end

Instance Method Details

#chunk_data(generator, **kwargs) ⇒ Object

Yields an object (can be anything) that will be passed to ‘upload_data_chunk` as `chunk`.

If multiple Destinations have the same ‘group_key`, `chunk_data` will only be called on the first and the chunk will be passed to each destination. Thus, if chunking is config-dependent, your Destination must modify the `group_key` to be unique for each configuration.

This must be overridden, but you may call super with a block to iterate individual rows. Manually batch the rows, or include Concerns::Chunking to pre-batch them.

Raises:

  • (NotImplementedError) 


28
29
30
31
32
33
34
35
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 28

def chunk_data(generator, **kwargs)
  raise NotImplementedError if method(__method__).owner == Base

  enum = Enumerator.new(&generator)
  enum.each do |row|
    yield format_row(row)
  end
end

#cleanup_fatal_error ⇒ Object

This method is called if a fatal error occurs. It should cleanup any external resources created by the dump.



49
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 49

def cleanup_fatal_error; end

#config ⇒ Object 



51
52
53
54
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 51

def config
  return @_config if @_config.is_a?(Hash)
  @config ||= parse_configuration(@_config)
end

#finalize_dump ⇒ Object

This method is called after processing all data. It should be used to finalize any external resources created by the dump.



45
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 45

def finalize_dump; end

#group_key ⇒ Object 



60
61
62
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 60

def group_key
  { class: self.class }
end

#initialize_dump(context) ⇒ Object

This method is called before processing any data. It should be used to initialize any external resources needed for the dump.



20
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 20

def initialize_dump(context); end

#preinitialize_dump(context) ⇒ Object

This method is called before taking any actions. It should be used to make any necessarry state assumptions (eg, the HostedData destination checks for a previous dump to determine if it can use incremental_since)



16
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 16

def preinitialize_dump(context); end

#upload_data_chunk(table_def, chunk) ⇒ Object

Called with any values yielded from chunk_data. This method should upload the chunk to the destination.

Raises:

  • (NotImplementedError) 


39
40
41
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 39

def upload_data_chunk(table_def, chunk)
  raise NotImplementedError
end

#user_config ⇒ Object 



56
57
58
 
# File 'lib/inst_data_shipper/destinations/base.rb', line 56

def user_config
  config[:user_config]
end