Class: Cascading::Tap

Inherits:
BaseTap show all
Defined in:
lib/cascading/tap.rb

Overview

A Tap represents a non-aggregate tap with a scheme, path, and optional sink_mode. c.t.l.FileTap is used in Cascading local mode and c.t.h.Hfs is used in Hadoop mode. Whether or not these can be created is governed by the :scheme parameter, which must contain at least one of :local_scheme or :hadoop_scheme. Schemes like TextLine are supported in both modes (by Cascading), but SequenceFile is only supported in Hadoop mode.

Instance Attribute Summary collapse

Attributes inherited from BaseTap

#hadoop_tap, #local_tap

Instance Method Summary collapse

Methods inherited from BaseTap

#hadoop?, #local?, #to_s

Constructor Details

#initialize(path, options = {}) ⇒ Tap

Builds a Tap given a required path

The named options are:

scheme

A Hash which must contain at least one of :local_scheme or :hadoop_scheme but may contain both. Default is text_line_scheme, which works in both modes.

sink_mode

A symbol or string that may be :keep, :replace, or :append, and corresponds to the c.t.SinkMode enumeration. The default is :keep, which matches Cascading’s default.



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/cascading/tap.rb', line 50

def initialize(path, options = {})
  @path = path

  @scheme = options[:scheme] || text_line_scheme
  raise "Scheme must provide one of :local_scheme or :hadoop_scheme; received: '#{scheme.inspect}'" unless scheme[:local_scheme] || scheme[:hadoop_scheme]

  @sink_mode = case options[:sink_mode] || :keep
    when :keep, 'keep'       then Java::CascadingTap::SinkMode::KEEP
    when :replace, 'replace' then Java::CascadingTap::SinkMode::REPLACE
    when :append, 'append'   then Java::CascadingTap::SinkMode::APPEND
    else raise "Unrecognized sink mode '#{options[:sink_mode]}'"
  end

  local_scheme = scheme[:local_scheme]
  @local_tap = local_scheme ? Java::CascadingTapLocal::FileTap.new(local_scheme, path, sink_mode) : nil

  hadoop_scheme = scheme[:hadoop_scheme]
  @hadoop_tap = hadoop_scheme ? Java::CascadingTapHadoop::Hfs.new(hadoop_scheme, path, sink_mode) : nil
end

Instance Attribute Details

#path ⇒ Object (readonly)

Returns the value of attribute path.



39
40
41
 
# File 'lib/cascading/tap.rb', line 39

def path
  @path
end

#scheme ⇒ Object (readonly)

Returns the value of attribute scheme.



39
40
41
 
# File 'lib/cascading/tap.rb', line 39

def scheme
  @scheme
end

#sink_mode ⇒ Object (readonly)

Returns the value of attribute sink_mode.



39
40
41
 
# File 'lib/cascading/tap.rb', line 39

def sink_mode
  @sink_mode
end