Module: Ductr::ETL::Parser

Included in:
Ductr::ETLJob, KibaJob
Defined in:
lib/ductr/etl/parser.rb

Overview

Contains anything to “parse” ETL jobs annotations. #parse_annotations handles ETL controls and send_to directives.

Instance Method Summary collapse

Instance Method Details

#adapters ⇒ Set

Currently used adapters set.

Returns:

  • (Set) —

    The current adapters



33
34
35
 
# File 'lib/ductr/etl/parser.rb', line 33

def adapters
  @adapters ||= Set.new
end

#parse_annotations ⇒ Array<Source, Transform, Destination, Hash{Symbol => Array<Symbol>}>

Handles sources, transforms and destinations controls. Handles send_to directives, used to do the plumbing between controls. Used for both kiba and fiber runners initialization.

Returns:



17
18
19
20
21
22
23
24
25
26
 
# File 'lib/ductr/etl/parser.rb', line 17

def parse_annotations
  sources = init_adapter_controls(:source)
  transforms = init_transform_controls(:transform, :lookup)
  destinations = init_adapter_controls(:destination)
  pipes = find_method(:send_to) do |method|
    { method.name => method.find_annotation(:send_to).params }
  end

  [sources, transforms, destinations, pipes]
end