Class: AWS::EMR::JobFlowCollection

Inherits:
Object
  • Object
show all
Includes:
Core::Collection::Simple
Defined in:
lib/aws/emr/job_flow_collection.rb

Overview

Creating a Job Flow

Call #create to run a new job flow.

emr = AWS::EMR.new

job_flow = emr.job_flows.create('name',
  :instances => {
    :instance_count => 2,
    :master_instance_type => 'm1.small',
    :slave_instance_type => 'm1.small',
  }
)

Getting a Job Flow

You can get a job flow by its ID.

job_flow = emr.job_flows['j-123456678'] # makes no request
job_flow.exists? #=> true/false

Enumerating Job Flows

You can enumerate all job flows, or filter them.

# all job flows
job_flows.each {|job_flow| ... }

# only job flows with a particular state
job_flows.with_state('ENDED').each {|job_flow| ... }

The filtering methods include:

Instance Method Summary collapse

Methods included from Core::Collection

#each, #each_batch, #enum, #first, #in_groups_of, #page

Instance Method Details

#[](job_flow_id) ⇒ JobFlow

Returns a AWS::EMR::JobFlow with the given ID.

Parameters:

  • job_flow_id (String)

Returns:



70
71
72
 
# File 'lib/aws/emr/job_flow_collection.rb', line 70

def [] job_flow_id
  JobFlow.new(job_flow_id, :config => config)
end

#create(name, options = {}) ⇒ JobFlow Also known as: run

Runs a job flow.

job_flow = emr.job_flows.create('name',
  :instances => {
    :instance_count => 2,
    :master_instance_type => 'm1.small',
    :slave_instance_type => 'm1.small',
  }
)

See Client#run_job_flow for documentation on the complete list of accepted options.

Parameters:

  • name (String)
  • options (Hash) (defaults to: {})

Returns:

See Also:

  • AWS::EMR::JobFlowCollection.(Client(Client#run_job_flow)


90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/aws/emr/job_flow_collection.rb', line 90

def create name, options = {}

  options[:name] = name
  options[:ami_version] ||= 'latest'
  options[:instances] ||= {}

  resp = client.run_job_flow(options)

  self[resp.data[:job_flow_id]]

end

#created_after(time) ⇒ JobFlowCollection

Returns a new collection that will only enumerate job flows that were created after the given time.

# enumerate jobs that are at most 1 hour old
emr.job_flows.created_after(Time.now - 3600).each{|job| ... }

Parameters:

  • time (Time, DateTime, Date)

Returns:



150
151
152
153
 
# File 'lib/aws/emr/job_flow_collection.rb', line 150

def created_after time
  time = time.iso8601 if time.respond_to?(:iso8601)
  filter(:created_after, time)
end

#created_before(time) ⇒ JobFlowCollection

Returns a new collection that will only enumerate job flows that were created before the given time.

# enumerate jobs that are more than an hour old
emr.job_flows.created_before(Time.now - 3600).each{|job| ... }

Parameters:

  • time (Time, DateTime, Date)

Returns:



137
138
139
140
 
# File 'lib/aws/emr/job_flow_collection.rb', line 137

def created_before time
  time = time.iso8601 if time.respond_to?(:iso8601)
  filter(:created_before, time)
end

#filter(name, value) ⇒ JobFlowCollection

Parameters:

  • name (String, Symbol)
  • value (Mixed)

Returns:



158
159
160
161
162
163
 
# File 'lib/aws/emr/job_flow_collection.rb', line 158

def filter name, value
  options = {}
  options[:filters] = @filters.merge(name.to_s.to_sym => value)
  options[:config] = config
  JobFlowCollection.new(options)
end

#with_id(*ids) ⇒ JobFlowCollection

Returns a new collection that will only enumerate job flows that have one of the given ids.

emr.job_flows.with_id('id1', 'id2', 'id3').each do |job_flow|
  # ...
end

Parameters:

  • ids (String) —

    One or more job flow ids to use as a filter.

Returns:



112
113
114
 
# File 'lib/aws/emr/job_flow_collection.rb', line 112

def with_id *ids
  filter(:job_flow_ids, ids.flatten)
end

#with_state(*states) ⇒ JobFlowCollection

Returns a new collection that will only enumerate job flows that have one of the given job flow states.

emr.job_flows.with_state('SHUTTING_DOWN', 'TERMINATED').each do |job|
  # ...
end

Parameters:

  • states (String) —

    One or more job flow states to use as a filter.

Returns:



125
126
127
 
# File 'lib/aws/emr/job_flow_collection.rb', line 125

def with_state *states
  filter(:job_flow_states, states.flatten)
end