Class: Cucloud::Ec2Utils

Inherits:
Object
  • Object
show all
Defined in:
lib/cucloud/ec2_utils.rb

Overview

EC2Utils class - anything ec2 related goes here!

Constant Summary collapse

UBUNTU_PATCH_COMMAND =

This is the command sent to ubuntu for patching

'apt-get update; apt-get -y upgrade; reboot'.freeze
AMAZON_PATCH_COMMAND =

This is the command sent to amazon linux machines for patching

'yum update -y; reboot & disown '.freeze
WAITER_MAX_ATTEMPS =

Max attemps for a waiter to try

240
WAITER_DELAY =

Delay between calls used by waiter to check status

15
TWO_WEEKS =

Two weeks in hours

336
DEFAULT_OS =

Default OS to use

'Linux/UNIX'.freeze

Instance Method Summary collapse

Constructor Details

#initialize(ec2_client = Aws::EC2::Client.new, ssm_utils = Cucloud::SSMUtils.new) ⇒ Ec2Utils

Returns a new instance of Ec2Utils.



17
18
19
20
 
# File 'lib/cucloud/ec2_utils.rb', line 17

def initialize(ec2_client = Aws::EC2::Client.new, ssm_utils = Cucloud::SSMUtils.new)
  @ec2 = ec2_client
  @ssm_utils = ssm_utils
end

Instance Method Details

#associate_eip(instance, allocation_id) ⇒ Object

Assoications an Elastic IP adress with a specific instance number.

Returns:

  • association_id as a string in the form of eipassoc-569cd631. This is the link between between the elastic network interface and the elastic IP address.



72
73
 
# File 'lib/cucloud/ec2_utils.rb', line 72

def associate_eip(instance, allocation_id)
end

#backup_volumes_unless_recent_backup(days = 5, preserve_volume_tags = [], additional_snapshot_tags = []) ⇒ Array<Hash>

Performs a backup on volumes that do not have a recent snapshot_info Tags specified in additional_snapshot_tags[] will take precedence over tags we would normally create or would have copied from the volume via preserve_tags[].

Parameters:

  • days (Integer) (defaults to: 5) —

    defaults to 5

  • preserve_volume_tags (Array) (defaults to: []) —

    Array of tag keys to copy from from volume, if present.

  • additional_snapshot_tags (Array) (defaults to: []) —

    Array of hashes containing additional tags to apply,

Returns:

  • (Array<Hash>) —

    An array of hashes containing snapshot_id, instance_name and volume



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
 
# File 'lib/cucloud/ec2_utils.rb', line 191

def backup_volumes_unless_recent_backup(days = 5, preserve_volume_tags = [], additional_snapshot_tags = [])
  volumes_backed_up_recently = volumes_with_snapshot_within_last_days(days)
  snapshots_created = []

  volumes = @ec2.describe_volumes(filters: [{ name: 'attachment.status', values: ['attached'] }])
  volumes.volumes.each do |volume|
    next if volumes_backed_up_recently[volume.volume_id.to_s]
    instance_name = get_instance_name(volume.attachments[0].instance_id)
    tags = additional_snapshot_tags.dup
    unless instance_name.nil? || tags.any? { |tagitem| tagitem[:key] == 'Instance Name' }
      tags << { key: 'Instance Name', value: instance_name }
    end
    volume.tags.each do |tag|
      if preserve_volume_tags.include?(tag.key) && !tags.any? { |tagitem| tagitem[:key] == tag.key }
        tags << tag
      end
    end

    snapshot_info = create_ebs_snapshot(volume.volume_id,
                                        'auto-ebs-snap-' + Time.now.strftime('%Y-%m-%d-%H:%M:%S'),
                                        tags)

    snapshots_created.push(snapshot_id: snapshot_info.snapshot_id,
                           instance_name: instance_name,
                           volume: volume.volume_id,
                           tags: tags)
  end

  snapshots_created
end

#best_spot_bid_price(instance_type, os = DEFAULT_OS, num_hours = TWO_WEEKS) ⇒ Hash

Get a recommendation for a spot bid request. Given an instance type and OS we will grab data from a period specified, default is from two weeks to now, and calculate recommendations for the AZs in the current region

Parameters:

  • instance_type (String) —

    Insrance type to get bid for

  • os (String) (defaults to: DEFAULT_OS) —

    OS you whish to run, default linux

  • num_hours (Integer) (defaults to: TWO_WEEKS) —

    How many hours to look back, default two weeks

Returns:

  • (Hash) —

    Reccomendations by region, empty if no viable recommendations



266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
 
# File 'lib/cucloud/ec2_utils.rb', line 266

def best_spot_bid_price(instance_type, os = DEFAULT_OS, num_hours = TWO_WEEKS)
  price_history_by_az = {}
  recommendations = {}

  options = {
    end_time: Time.now.utc,
    instance_types: [
      instance_type
    ],
    product_descriptions: [
      os
    ],
    start_time: (Time.now - num_hours * 60).utc
  }

  loop do
    price_history = @ec2.describe_spot_price_history(options)
    price_history.spot_price_history.each do |price|
      price_history_by_az[price.availability_zone] = [] unless price_history_by_az[price.availability_zone]
      price_history_by_az[price.availability_zone].push(price.spot_price.to_f)
    end

    break if price_history.next_token.nil? || price_history.next_token.empty?
    options[:next_token] = price_history.next_token
  end

  price_history_by_az.each do |key, data|
    stats = data.descriptive_statistics
    next unless stats[:number] > 30
    confidence_interval = Cucloud::Utilities.confidence_interval_99(
      stats[:mean],
      stats[:standard_deviation],
      stats[:number]
    )
    recommendations[key] = confidence_interval[1]
  end

  recommendations
end

#create_ebs_snapshot(volume_id, snapshot_desc, tags = []) ⇒ Object

Create a snapshot of an EBS volume and apply supplied tags will wait 20 minutes for the process to completed http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Client.html#create_tags-instance_method

Parameters:

  • volume_id (String) —

    volume id in the formate of vol-121231231231

  • snapshot_desc (String) —

    Description of the snapshot

  • tags (Array) (defaults to: []) —

    Array of key value pairs to be applied as tags to the snapshot

Returns:

  • snapshot information see



168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/cucloud/ec2_utils.rb', line 168

def create_ebs_snapshot(volume_id, snapshot_desc, tags = [])
  snapshot_info = @ec2.create_snapshot(
    volume_id: volume_id,
    description: snapshot_desc
  )

  @ec2.wait_until(:snapshot_completed, snapshot_ids: [snapshot_info.snapshot_id]) do |w|
    w.max_attempts = WAITER_MAX_ATTEMPS
    w.delay = WAITER_DELAY
  end

  @ec2.create_tags(resources: [snapshot_info.snapshot_id], tags: tags) unless tags.empty?

  snapshot_info
end

#create_instance(options) ⇒ Object

Create ec2 instance based on parameters provided. The function will pull in default information from ?????.

Parameters:

  • options (hash) —

    will be hash that will override the default



78
79
 
# File 'lib/cucloud/ec2_utils.rb', line 78

def create_instance(options)
end

#deregister_image(image) ⇒ Object

Remove private AMI



82
83
 
# File 'lib/cucloud/ec2_utils.rb', line 82

def deregister_image(image)
end

#find_ami(name) ⇒ Object

Find ami based on a search of Name



86
87
 
# File 'lib/cucloud/ec2_utils.rb', line 86

def find_ami(name)
end

#find_ebs_snapshots(options = {}) ⇒ Array

Find snapshots with supplied properties, currently only supports days_old

Parameters:

  • options (Hash) (defaults to: {})

Returns:

  • (Array) —

    list of snapshot ids



240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
 
# File 'lib/cucloud/ec2_utils.rb', line 240

def find_ebs_snapshots(options = {})
  days_old = options[:days_old]
  found_snapshots = []
  snapshots = @ec2.describe_snapshots(owner_ids: ['self'], filters: [{ name: 'status', values: ['completed'] }])

  snapshots.snapshots.each do |snapshot|
    if !days_old.nil?
      snapshot_days_old = (Time.now.to_i - snapshot.start_time.to_i) / Cucloud::SECONDS_IN_A_DAY

      if snapshot_days_old > days_old
        found_snapshots.push(snapshot.snapshot_id)
      end
    else
      found_snapshots.push(snapshot.snapshot_id)
    end
  end
  found_snapshots
end

#get_instance(instance_id) ⇒ Aws::EC2::Instance

Get instance object http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Instance.html

Parameters:

  • instance_id (String) —

    instance id in the format of i-121231231231

Returns:

  • (Aws::EC2::Instance) —

    Object representing the intance see



26
27
28
 
# File 'lib/cucloud/ec2_utils.rb', line 26

def get_instance(instance_id)
  Aws::EC2::Instance.new(id: instance_id, client: @ec2)
end

#get_instance_information(instance) ⇒ array

Get instance information for a specific instance

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231

Returns:



34
35
36
 
# File 'lib/cucloud/ec2_utils.rb', line 34

def get_instance_information(instance)
  @ec2.describe_instances(instance_ids: [instance])
end

#get_instance_name(instance_id) ⇒ String

Get the nice name of the ec2 intsance from the 'Name" tag'

Parameters:

  • instance_id (String) —

    instance id in the format of i-121231231231

Returns:

  • (String) —

    Name of instacnce if found or nil if not found



155
156
157
158
159
 
# File 'lib/cucloud/ec2_utils.rb', line 155

def get_instance_name(instance_id)
  instance = get_instance(instance_id)
  tag_name = instance.tags.find { |tag| tag.key.eql?('Name') }
  tag_name ? tag_name.value : nil
end

#get_instances_by_tag(tag_name, tag_value) ⇒ array

Based on tag name and value, return instances

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag

Returns:



94
95
96
97
98
99
100
101
 
# File 'lib/cucloud/ec2_utils.rb', line 94

def get_instances_by_tag(tag_name, tag_value)
  @ec2.describe_instances(filters: [
                            {
                              name: "tag:#{tag_name}",
                              values: tag_value
                            }
                          ])
end

#instances_to_patch_by_tag(tag_name = 'auto_patch', tag_value = ['1']) ⇒ Object

patch instances based on a tag name and value

Parameters:

  • tag_name (string) (defaults to: 'auto_patch') —

    name of tag

  • tag_value (string) (defaults to: ['1']) —

    the value of the tag



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/cucloud/ec2_utils.rb', line 124

def instances_to_patch_by_tag(tag_name = 'auto_patch', tag_value = ['1'])
  resp = get_instances_by_tag(tag_name, tag_value)

  ubuntu_patch_instances = []
  amazon_patch_instances = []
  all_instances = []

  resp.reservations.each do |res|
    res.instances.each do |instance|
      instance.tags.each do |tag|
        next unless tag.key.eql?('os')
        if tag.value.eql?('ubuntu')
          ubuntu_patch_instances.push(instance.instance_id)
          all_instances.push(instance.instance_id)
        elsif tag.value.eql?('ecs') || tag.value.eql?('amazon')
          amazon_patch_instances.push(instance.instance_id)
          all_instances.push(instance.instance_id)
        end
      end
    end
  end

  @ssm_utils.send_patch_command(ubuntu_patch_instances, UBUNTU_PATCH_COMMAND) if ubuntu_patch_instances.any?
  @ssm_utils.send_patch_command(amazon_patch_instances, AMAZON_PATCH_COMMAND) if amazon_patch_instances.any?

  all_instances
end

#make_spot_instance_request(options) ⇒ Hash

Make spot instance request see http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Client.html#request_spot_instances-instance_method

Parameters:

  • options (Hash) —

    Options to provide to the API

Returns:

  • (Hash) —

    Description of the spot request



310
311
312
313
314
315
316
 
# File 'lib/cucloud/ec2_utils.rb', line 310

def make_spot_instance_request(options)
  spot_requests = @ec2.request_spot_instances(options)
  request_ids = [spot_requests.spot_instance_requests[0].spot_instance_request_id]

  @ec2.wait_until(:spot_instance_request_fulfilled, spot_instance_request_ids: request_ids)
  @ec2.describe_spot_instance_requests(spot_instance_request_ids: request_ids)
end

#reboot_instance(instance) ⇒ Object

reboot instance



57
58
59
60
 
# File 'lib/cucloud/ec2_utils.rb', line 57

def reboot_instance(instance)
  i = get_instance(instance)
  i.reboot
end

#rename_instance(instance, name) ⇒ Object

Set the name of the instance that will be displayed in the ec2 console



53
54
 
# File 'lib/cucloud/ec2_utils.rb', line 53

def rename_instance(instance, name)
end

#start_instance(instance) ⇒ Object

Start ec2 instance for a specific instance number. The function will wait until the instance has entered the running state.

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231



48
49
50
 
# File 'lib/cucloud/ec2_utils.rb', line 48

def start_instance(instance)
  @ec2.start_instances(instance_ids: [instance])
end

#start_instances_by_tag(tag_name, tag_value) ⇒ Object

start instances based on a tag name and value

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag



115
116
117
118
119
 
# File 'lib/cucloud/ec2_utils.rb', line 115

def start_instances_by_tag(tag_name, tag_value)
  get_instances_by_tag(tag_name, tag_value).reservations[0].instances.each do |i|
    @ec2.start_instances(instance_ids: [i.instance_id])
  end
end

#stop_instance(instance) ⇒ Object

Stop ec2 instance for a specific instance number. The function will wait until the instance has entered the stopped state.

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231



41
42
43
 
# File 'lib/cucloud/ec2_utils.rb', line 41

def stop_instance(instance)
  @ec2.stop_instances(instance_ids: [instance])
end

#stop_instances_by_tag(tag_name, tag_value) ⇒ Object

stop instances based on a tag name and value

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag



106
107
108
109
110
 
# File 'lib/cucloud/ec2_utils.rb', line 106

def stop_instances_by_tag(tag_name, tag_value)
  get_instances_by_tag(tag_name, tag_value).reservations[0].instances.each do |i|
    @ec2.stop_instances(instance_ids: [i.instance_id])
  end
end

#terminate_instance(instance) ⇒ Object

Terminate ec2 instance for a specific instance number.



63
64
65
66
 
# File 'lib/cucloud/ec2_utils.rb', line 63

def terminate_instance(instance)
  i = get_instance(instance)
  i.terminate
end

#volumes_with_snapshot_within_last_days(days = 5) ⇒ Array

Find volumes that have a recent snapshot

Parameters:

  • days (Integer) (defaults to: 5) —

    defaults to 5

Returns:

  • (Array) —

    list of volume ids that have recent snapshots



225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/cucloud/ec2_utils.rb', line 225

def volumes_with_snapshot_within_last_days(days = 5)
  volumes_backed_up_recently = {}

  snapshots = @ec2.describe_snapshots(owner_ids: ['self'], filters: [{ name: 'status', values: ['completed'] }])
  snapshots.snapshots.each do |snapshot|
    if snapshot.start_time > Time.now - (Cucloud::SECONDS_IN_A_DAY * days)
      volumes_backed_up_recently[snapshot.volume_id.to_s] = true
    end
  end
  volumes_backed_up_recently
end