Class: Stockboy::Providers::File

Inherits:
Stockboy::Provider show all
Defined in:
lib/stockboy/providers/file.rb

Overview

Get data from a local file

Allows for selecting the appropriate file to be read from the given directory by glob pattern or regex pattern. By default the :last file in the list is used, but can be controlled by sorting and reducing with the #pick option.

Job template DSL

provider :file do
  file_dir '/data'
  file_name /report-[0-9]+\.csv/
  pick ->(list) { list[-2] }
end

Options collapse

Attributes inherited from Stockboy::Provider

#data, #data_size, #data_time, #errors, #logger

Instance Method Summary collapse

Methods inherited from Stockboy::Provider

#data?, #inspect, #reload, #valid?

Constructor Details

#initialize(opts = {}, &block) ⇒ File

Initialize a File provider



40
41
42
43
44
45
46
47
48
49
 
# File 'lib/stockboy/providers/file.rb', line 40

def initialize(opts={}, &block)
  super(opts, &block)
  @file_dir     = opts[:file_dir]
  @file_name    = opts[:file_name]
  @file_newer   = opts[:file_newer]
  @file_smaller = opts[:file_smaller]
  @file_larger  = opts[:file_larger]
  @pick         = opts[:pick] || :last
  DSL.new(self).instance_eval(&block) if block_given?
end

Instance Attribute Details

#file_dir ⇒ String

Path where data files can be found. This should be an absolute path.

Examples:

file_dir "/data"

Returns:

  • (String) 


25
 
# File 'lib/stockboy/providers/file.rb', line 25

dsl_attr :file_name

#file_larger ⇒ Integer

Validates the minimum file size for the matched file, in bytes. This can # help guard against processing zero-byte or truncated files. This validation option is applied after a matching file is picked.

Examples:

file_larger 1024

Returns:

  • (Integer) 


30
 
# File 'lib/stockboy/providers/file.rb', line 30

dsl_attr :file_smaller

#file_name ⇒ String, Regexp

A string (glob) or regular expression matching files. E.g. one of:

Examples:

file_name "export-latest.csv"
file_name "export-*.csv"
file_name /export-\d{4}-\d{2}-\d{2}.csv/

Returns:

  • (String, Regexp) 


25
 
# File 'lib/stockboy/providers/file.rb', line 25

dsl_attr :file_name

#file_newer ⇒ Time, Date

Validates that the file to be processed is recent enough. To guard against processing an old file (even if it’s the latest one), this should be set to the frequency you expect to receive new files for periodic processing. This validation option is applied after a matching file is picked.

Examples:

since Date.today

Returns:

  • (Time, Date) 


25
 
# File 'lib/stockboy/providers/file.rb', line 25

dsl_attr :file_name

#file_smaller ⇒ Integer

Validates the maximum data size for the matched file, in bytes This validation option is applied after a matching file is picked.

Examples:

file_smaller 1024^3

Returns:

  • (Integer) 


30
 
# File 'lib/stockboy/providers/file.rb', line 30

dsl_attr :file_smaller

#pick ⇒ Object

Method for choosing which file to process from potential matches.

@example
  pick :last
  pick :first
  pick ->(list) {
    list.max_by { |name| Time.strptime(name[/\d+/], "%m%d%Y").to_i }
  }



34
 
# File 'lib/stockboy/providers/file.rb', line 34

dsl_attr :pick

Instance Method Details

#clear ⇒ Object 



62
63
64
65
66
67
 
# File 'lib/stockboy/providers/file.rb', line 62

def clear
  super
  @matching_file = nil
  @data_size = nil
  @data_time = nil
end

#delete_data ⇒ Object

Raises:



51
52
53
54
55
56
 
# File 'lib/stockboy/providers/file.rb', line 51

def delete_data
  raise Stockboy::OutOfSequence, "must confirm #matching_file or calling #data" unless picked_matching_file?

  logger.info "Deleting file #{file_dir}/#{matching_file}"
  ::File.delete matching_file
end

#matching_file ⇒ Object 



58
59
60
 
# File 'lib/stockboy/providers/file.rb', line 58

def matching_file
  @matching_file ||= pick_from(file_list.sort)
end