Class: UniqueIndexedSumReduce

Inherits:

ReduceBase

• Object
• Stage
• ReduceBase
• UniqueIndexedSumReduce

Defined in:

lib/mrtoolkit.rb

Overview

Reducer works on groups where the first field is the same. For each distinct value of the second field, sum up the values

of the third field.

Instance Attribute Summary

Attributes inherited from Stage

#errors, #in_fields, #in_sep, #out_fields, #out_sep

Instance Method Summary

• #declare ⇒ Object
• #process_each(input, output) ⇒ Object
• #process_init(input, output) ⇒ Object
• #process_term(dummy, output) ⇒ Object

Methods inherited from ReduceBase

#process, #process_begin, #process_end, #process_end_internal, #process_internal, #run

Methods inherited from Stage

#catch_errors, #copy_struct, #emit, #emit_separator, #field, #field_separator, #initialize, #new_input, #new_output, #prepare, #process_step, #write_out

Constructor Details

This class inherits a constructor from Stage

Instance Method Details

#declare ⇒ Object

# File 'lib/mrtoolkit.rb', line 460

def declare
  field :unique
  field :index
  field :value

  emit :unique
  emit :index
  emit :value
end

#process_each(input, output) ⇒ Object

# File 'lib/mrtoolkit.rb', line 473

def process_each(input, output)
  index = input.index
  @sum[index] = 0 unless @sum.has_key?(index) 
  @sum[index] += input.value.to_i 
  nil
end

#process_init(input, output) ⇒ Object

# File 'lib/mrtoolkit.rb', line 469

def process_init(input, output)
  @sum = {}
  nil
end

#process_term(dummy, output) ⇒ Object

# File 'lib/mrtoolkit.rb', line 479

def process_term(dummy, output)
  output = []
  @sum.each do |index, value|
    item = new_output
    item.unique = @last
    item.index = index
    item.value = value
    output << item
  end
  output
end