Class: Stockboy::Configurator

Inherits:
Object
  • Object
show all
Defined in:
lib/stockboy/configurator.rb

Overview

Context for evaluating DSL templates and capturing job options for initializing a job.

Wraps up the DSL methods called in job templates and handles the construction of the job’s provider, reader, attributes, and filters.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#new(dsl, file = ) ⇒ Configurator #new(&block) ⇒ Configurator

Evaluate DSL and capture configuration for building a job

Overloads:

  • #new(dsl, file = ) ⇒ Configurator

    Evaluate DSL from a string

    Parameters:

    • dsl (String) —

      job template language for evaluation

    • file (String) (defaults to: ) —

      path to original file for reporting errors

  • #new(&block) ⇒ Configurator

    Evaluate DSL in a block



32
33
34
35
36
37
38
39
40
41
 
# File 'lib/stockboy/configurator.rb', line 32

def initialize(dsl='', file=__FILE__, &block)
  @config = {}
  @config[:triggers] = Hash.new { |hash, key| hash[key] = [] }
  @config[:filters] = {}
  if block_given?
    instance_eval(&block)
  else
    instance_eval(dsl, file)
  end
end

Instance Attribute Details

#config ⇒ Hash (readonly)

Captured job configuration options

Returns:

  • (Hash) 


21
22
23
 
# File 'lib/stockboy/configurator.rb', line 21

def config
  @config
end

Instance Method Details

#attribute(key, opts = {}) ⇒ Object

Add individual attribute mapping rules

Parameters:

  • key (Symbol) —

    Name of the output attribute

  • opts (Hash) (defaults to: {})

Options Hash (opts):

  • from (String) —

    Name of input field from reader

  • as (Array, Proc, Translator) —

    One or more translators



144
145
146
147
 
# File 'lib/stockboy/configurator.rb', line 144

def attribute(key, opts={})
  @config[:attributes] ||= AttributeMap.new
  @config[:attributes].insert(key, opts)
end

#attributes(&block) ⇒ Object

Configure the attribute map for data records

This will replace any existing attributes with a new set.

Examples:

attributes do
  first_name as: ->(raw){ raw["FullName"].split(" ").first }
  email      from: "RawEmail", as: [:string]
  check_in   from: "RawCheckIn", as: [:date]
end

Raises:

  • (ArgumentError) 


130
131
132
133
134
 
# File 'lib/stockboy/configurator.rb', line 130

def attributes(&block)
  raise ArgumentError unless block_given?

  @config[:attributes] = AttributeMap.new(&block)
end

#filter(key, callable = nil, *args, &block) ⇒ Object

Add a filter to the processing filter chain

  • Must be called with either a callable argument (proc) or a block.

  • Must be called in the order that filters should be applied.

Examples:

filter :missing_email do |raw, out|
  raw["RawEmail"].empty?
end
filter :past_due do |raw, out|
  out.check_in < Date.today
end
filter :under_age, :check_id
filter :update, proc{ true } # capture all remaining items


164
165
166
167
168
 
# File 'lib/stockboy/configurator.rb', line 164

def filter(key, callable=nil, *args, &block)
  filter = Filters.build(callable, args, block) || block
  filter or raise ArgumentError, "Missing filter arguments for #{key}"
  @config[:filters][key] = filter
end

#on(key) {|, Arguments| ... } ⇒ Object

Register a trigger to notify the job of external events

Useful for adding generic control over the job’s resources from your app. For example, if you need to record stats or clean up data after your application has successfully processed the records, these actions can be defined within the context of each job template.

Examples:

trigger :cleanup do |job, *args|
  job.provider.delete_data
end

# elsewhere:
if MyProjects.find(123).import_records(job.records[:valid])
  job.cleanup
end

Parameters:

  • key (Symbol) —

    Name of the trigger

Yield Parameters:

  • (Stockboy::Job)
  • Arguments (Array) —

    passed to the action when called

Raises:

  • (ArgumentError) 


191
192
193
194
195
 
# File 'lib/stockboy/configurator.rb', line 191

def on(key, &block)
  raise(ArgumentError, "no block given") unless block_given?

  @config[:triggers][key] << block
end

#provider(key, opts = {}, &block) ⇒ Provider Also known as: connection

Configure the provider for fetching data

The optional block is evaluated in the provider’s own DSL context.

Examples:

provider :file, file_dir: "/downloads/@client" do
  file_name "example.csv"
end

Parameters:

  • key (Symbol, Class, Provider) —

    The registered symbol name for the provider, or actual provider

  • opts (Hash) (defaults to: {}) —

    Provider-specific options passed to the provider initializer

Returns:



59
60
61
 
# File 'lib/stockboy/configurator.rb', line 59

def provider(key, opts={}, &block)
  @config[:provider] = Providers.build(key, opts, block)
end

#reader(key, opts = {}, &block) ⇒ Reader Also known as: format

Configure the reader for parsing data

Examples:

reader :csv do
  col_sep "|"
end

Parameters:

  • key (Symbol, Class, Reader) —

    The registered symbol name for the reader, or actual reader instance

  • opts (Hash) (defaults to: {}) —

    Provider-specific options passed to the provider initializer

Returns:



114
115
116
 
# File 'lib/stockboy/configurator.rb', line 114

def reader(key, opts={}, &block)
  @config[:reader] = Readers.build(key, opts, block)
end

#repeat(&block) ⇒ Object

Configure repeating the provider for fetching multiple parts

If the provider needs to give us all the data as a series of requests, for example multiple HTTP pages or FTP files, the repeat block can be used to define the iteration for fetching each item.

The ‘<<` interface used here is defined by Ruby’s Enumerator.new block syntax. For each page that needs to be fetched, the provider options need to be altered and pushed on to the output. Control will be yielded to the reader at each iteration.

Examples:

repeat do |output, provider|
  loop do
    output << provider
    break if provider.data.split("\n").size < 100
    provider.query_params["page"] += 1
  end
end
repeat do |output, provider|
  1.upto 10 do |i|
    provider.file_name = "example-#{i}.log"
    output << provider
  end
end


92
93
94
95
96
97
98
 
# File 'lib/stockboy/configurator.rb', line 92

def repeat(&block)
  unless block_given? && block.arity == 2
    raise ArgumentError, "repeat block must accept |output, provider| arguments"
  end

  @config[:repeat] = block
end

#to_job ⇒ Job

Initialize a new job with the captured options

Returns:



201
202
203
 
# File 'lib/stockboy/configurator.rb', line 201

def to_job
  Job.new(config_for_job)
end