Class: Cucloud::Ec2Utils

Inherits:
Object
  • Object
show all
Defined in:
lib/cucloud/ec2_utils.rb

Overview

EC2Utils class - anything ec2 related goes here!

Constant Summary collapse

UBUNTU_PATCH_COMMAND =

This is the command sent to ubuntu for patching

'apt-get update; apt-get -y upgrade; reboot'.freeze
AMAZON_PATCH_COMMAND =

This is the command sent to amazon linux machines for patching

'yum update -y; reboot & disown '.freeze
SECONDS_IN_A_DAY =

Used in time calculations

86_400
WAITER_MAX_ATTEMPS =

Max attemps for a waiter to try

240
WAITER_DELAY =

Delay between calls used by waiter to check status

15
TWO_WEEKS =

Two weeks in hours

336
DEFAULT_OS =

Default OS to use

'Linux/UNIX'.freeze

Instance Method Summary collapse

Constructor Details

#initialize(ec2_client = Aws::EC2::Client.new, ssm_utils = Cucloud::SSMUtils.new) ⇒ Ec2Utils

Returns a new instance of Ec2Utils.



19
20
21
22
 
# File 'lib/cucloud/ec2_utils.rb', line 19

def initialize(ec2_client = Aws::EC2::Client.new, ssm_utils = Cucloud::SSMUtils.new)
  @ec2 = ec2_client
  @ssm_utils = ssm_utils
end

Instance Method Details

#associate_eip(instance, allocation_id) ⇒ Object

Assoications an Elastic IP adress with a specific instance number.

Returns:

  • association_id as a string in the form of eipassoc-569cd631. This is the link between between the elastic network interface and the elastic IP address.



74
75
 
# File 'lib/cucloud/ec2_utils.rb', line 74

def associate_eip(instance, allocation_id)
end

#backup_volumes_unless_recent_backup(days = 5, preserve_volume_tags = [], additional_snapshot_tags = []) ⇒ Array<Hash>

Performs a backup on volumes that do not have a recent snapshot_info Tags specified in additional_snapshot_tags[] will take precedence over tags we would normally create or would have copied from the volume via preserve_tags[].

Parameters:

  • days (Integer) (defaults to: 5) —

    defaults to 5

  • preserve_volume_tags (Array) (defaults to: []) —

    Array of tag keys to copy from from volume, if present.

  • additional_snapshot_tags (Array) (defaults to: []) —

    Array of hashes containing additional tags to apply,

Returns:

  • (Array<Hash>) —

    An array of hashes containing snapshot_id, instance_name and volume



193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/cucloud/ec2_utils.rb', line 193

def backup_volumes_unless_recent_backup(days = 5, preserve_volume_tags = [], additional_snapshot_tags = [])
  volumes_backed_up_recently = volumes_with_snapshot_within_last_days(days)
  snapshots_created = []

  volumes = @ec2.describe_volumes(filters: [{ name: 'attachment.status', values: ['attached'] }])
  volumes.volumes.each do |volume|
    next if volumes_backed_up_recently[volume.volume_id.to_s]
    instance_name = get_instance_name(volume.attachments[0].instance_id)
    tags = additional_snapshot_tags.dup
    unless instance_name.nil? || tags.any? { |tagitem| tagitem[:key] == 'Instance Name' }
      tags << { key: 'Instance Name', value: instance_name }
    end
    volume.tags.each do |tag|
      if preserve_volume_tags.include?(tag.key) && !tags.any? { |tagitem| tagitem[:key] == tag.key }
        tags << tag
      end
    end

    snapshot_info = create_ebs_snapshot(volume.volume_id,
                                        'auto-ebs-snap-' + Time.now.strftime('%Y-%m-%d-%H:%M:%S'),
                                        tags)

    snapshots_created.push(snapshot_id: snapshot_info.snapshot_id,
                           instance_name: instance_name,
                           volume: volume.volume_id,
                           tags: tags)
  end

  snapshots_created
end

#best_spot_bid_price(instance_type, os = DEFAULT_OS, num_hours = TWO_WEEKS) ⇒ Hash

Get a recommendation for a spot bid request. Given an instance type and OS we will grab data from a period specified, default is from two weeks to now, and calculate recommendations for the AZs in the current region

Parameters:

  • instance_type (String) —

    Insrance type to get bid for

  • os (String) (defaults to: DEFAULT_OS) —

    OS you whish to run, default linux

  • num_hours (Integer) (defaults to: TWO_WEEKS) —

    How many hours to look back, default two weeks

Returns:

  • (Hash) —

    Reccomendations by region, empty if no viable recommendations



268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
 
# File 'lib/cucloud/ec2_utils.rb', line 268

def best_spot_bid_price(instance_type, os = DEFAULT_OS, num_hours = TWO_WEEKS)
  price_history_by_az = {}
  recommendations = {}

  options = {
    end_time: Time.now.utc,
    instance_types: [
      instance_type
    ],
    product_descriptions: [
      os
    ],
    start_time: (Time.now - num_hours * 60).utc
  }

  loop do
    price_history = @ec2.describe_spot_price_history(options)
    price_history.spot_price_history.each do |price|
      price_history_by_az[price.availability_zone] = [] unless price_history_by_az[price.availability_zone]
      price_history_by_az[price.availability_zone].push(price.spot_price.to_f)
    end

    break if price_history.next_token.nil? || price_history.next_token.empty?
    options[:next_token] = price_history.next_token
  end

  price_history_by_az.each do |key, data|
    stats = data.descriptive_statistics
    next unless stats[:number] > 30
    confidence_interval = Cucloud::Utilities.confidence_interval_99(
      stats[:mean],
      stats[:standard_deviation],
      stats[:number]
    )
    recommendations[key] = confidence_interval[1]
  end

  recommendations
end

#create_ebs_snapshot(volume_id, snapshot_desc, tags = []) ⇒ Object

Create a snapshot of an EBS volume and apply supplied tags will wait 20 minutes for the process to completed http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Client.html#create_tags-instance_method

Parameters:

  • volume_id (String) —

    volume id in the formate of vol-121231231231

  • snapshot_desc (String) —

    Description of the snapshot

  • tags (Array) (defaults to: []) —

    Array of key value pairs to be applied as tags to the snapshot

Returns:

  • snapshot information see



170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/cucloud/ec2_utils.rb', line 170

def create_ebs_snapshot(volume_id, snapshot_desc, tags = [])
  snapshot_info = @ec2.create_snapshot(
    volume_id: volume_id,
    description: snapshot_desc
  )

  @ec2.wait_until(:snapshot_completed, snapshot_ids: [snapshot_info.snapshot_id]) do |w|
    w.max_attempts = WAITER_MAX_ATTEMPS
    w.delay = WAITER_DELAY
  end

  @ec2.create_tags(resources: [snapshot_info.snapshot_id], tags: tags) unless tags.empty?

  snapshot_info
end

#create_instance(options) ⇒ Object

Create ec2 instance based on parameters provided. The function will pull in default information from ?????.

Parameters:

  • options (hash) —

    will be hash that will override the default



80
81
 
# File 'lib/cucloud/ec2_utils.rb', line 80

def create_instance(options)
end

#deregister_image(image) ⇒ Object

Remove private AMI



84
85
 
# File 'lib/cucloud/ec2_utils.rb', line 84

def deregister_image(image)
end

#find_ami(name) ⇒ Object

Find ami based on a search of Name



88
89
 
# File 'lib/cucloud/ec2_utils.rb', line 88

def find_ami(name)
end

#find_ebs_snapshots(options = {}) ⇒ Array

Find snapshots with supplied properties, currently only supports days_old

Parameters:

  • options (Hash) (defaults to: {})

Returns:

  • (Array) —

    list of snapshot ids



242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
 
# File 'lib/cucloud/ec2_utils.rb', line 242

def find_ebs_snapshots(options = {})
  days_old = options[:days_old]
  found_snapshots = []
  snapshots = @ec2.describe_snapshots(owner_ids: ['self'], filters: [{ name: 'status', values: ['completed'] }])

  snapshots.snapshots.each do |snapshot|
    if !days_old.nil?
      snapshot_days_old = (Time.now.to_i - snapshot.start_time.to_i) / SECONDS_IN_A_DAY

      if snapshot_days_old > days_old
        found_snapshots.push(snapshot.snapshot_id)
      end
    else
      found_snapshots.push(snapshot.snapshot_id)
    end
  end
  found_snapshots
end

#get_instance(instance_id) ⇒ Aws::EC2::Instance

Get instance object http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Instance.html

Parameters:

  • instance_id (String) —

    instance id in the format of i-121231231231

Returns:

  • (Aws::EC2::Instance) —

    Object representing the intance see



28
29
30
 
# File 'lib/cucloud/ec2_utils.rb', line 28

def get_instance(instance_id)
  Aws::EC2::Instance.new(id: instance_id, client: @ec2)
end

#get_instance_information(instance) ⇒ array

Get instance information for a specific instance

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231

Returns:



36
37
38
 
# File 'lib/cucloud/ec2_utils.rb', line 36

def get_instance_information(instance)
  @ec2.describe_instances(instance_ids: [instance])
end

#get_instance_name(instance_id) ⇒ String

Get the nice name of the ec2 intsance from the 'Name" tag'

Parameters:

  • instance_id (String) —

    instance id in the format of i-121231231231

Returns:

  • (String) —

    Name of instacnce if found or nil if not found



157
158
159
160
161
 
# File 'lib/cucloud/ec2_utils.rb', line 157

def get_instance_name(instance_id)
  instance = get_instance(instance_id)
  tag_name = instance.tags.find { |tag| tag.key.eql?('Name') }
  tag_name ? tag_name.value : nil
end

#get_instances_by_tag(tag_name, tag_value) ⇒ array

Based on tag name and value, return instances

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag

Returns:



96
97
98
99
100
101
102
103
 
# File 'lib/cucloud/ec2_utils.rb', line 96

def get_instances_by_tag(tag_name, tag_value)
  @ec2.describe_instances(filters: [
                            {
                              name: "tag:#{tag_name}",
                              values: tag_value
                            }
                          ])
end

#instances_to_patch_by_tag(tag_name = 'auto_patch', tag_value = ['1']) ⇒ Object

patch instances based on a tag name and value

Parameters:

  • tag_name (string) (defaults to: 'auto_patch') —

    name of tag

  • tag_value (string) (defaults to: ['1']) —

    the value of the tag



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
 
# File 'lib/cucloud/ec2_utils.rb', line 126

def instances_to_patch_by_tag(tag_name = 'auto_patch', tag_value = ['1'])
  resp = get_instances_by_tag(tag_name, tag_value)

  ubuntu_patch_instances = []
  amazon_patch_instances = []
  all_instances = []

  resp.reservations.each do |res|
    res.instances.each do |instance|
      instance.tags.each do |tag|
        next unless tag.key.eql?('os')
        if tag.value.eql?('ubuntu')
          ubuntu_patch_instances.push(instance.instance_id)
          all_instances.push(instance.instance_id)
        elsif tag.value.eql?('ecs') || tag.value.eql?('amazon')
          amazon_patch_instances.push(instance.instance_id)
          all_instances.push(instance.instance_id)
        end
      end
    end
  end

  @ssm_utils.send_patch_command(ubuntu_patch_instances, UBUNTU_PATCH_COMMAND) if ubuntu_patch_instances.any?
  @ssm_utils.send_patch_command(amazon_patch_instances, AMAZON_PATCH_COMMAND) if amazon_patch_instances.any?

  all_instances
end

#make_spot_instance_request(options) ⇒ Hash

Make spot instance request see http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/Client.html#request_spot_instances-instance_method

Parameters:

  • options (Hash) —

    Options to provide to the API

Returns:

  • (Hash) —

    Description of the spot request



312
313
314
315
316
317
318
 
# File 'lib/cucloud/ec2_utils.rb', line 312

def make_spot_instance_request(options)
  spot_requests = @ec2.request_spot_instances(options)
  request_ids = [spot_requests.spot_instance_requests[0].spot_instance_request_id]

  @ec2.wait_until(:spot_instance_request_fulfilled, spot_instance_request_ids: request_ids)
  @ec2.describe_spot_instance_requests(spot_instance_request_ids: request_ids)
end

#reboot_instance(instance) ⇒ Object

reboot instance



59
60
61
62
 
# File 'lib/cucloud/ec2_utils.rb', line 59

def reboot_instance(instance)
  i = get_instance(instance)
  i.reboot
end

#rename_instance(instance, name) ⇒ Object

Set the name of the instance that will be displayed in the ec2 console



55
56
 
# File 'lib/cucloud/ec2_utils.rb', line 55

def rename_instance(instance, name)
end

#start_instance(instance) ⇒ Object

Start ec2 instance for a specific instance number. The function will wait until the instance has entered the running state.

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231



50
51
52
 
# File 'lib/cucloud/ec2_utils.rb', line 50

def start_instance(instance)
  @ec2.start_instances(instance_ids: [instance])
end

#start_instances_by_tag(tag_name, tag_value) ⇒ Object

start instances based on a tag name and value

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag



117
118
119
120
121
 
# File 'lib/cucloud/ec2_utils.rb', line 117

def start_instances_by_tag(tag_name, tag_value)
  get_instances_by_tag(tag_name, tag_value).reservations[0].instances.each do |i|
    @ec2.start_instances(instance_ids: [i.instance_id])
  end
end

#stop_instance(instance) ⇒ Object

Stop ec2 instance for a specific instance number. The function will wait until the instance has entered the stopped state.

Parameters:

  • instance (String) —

    instance id in the format of i-121231231231



43
44
45
 
# File 'lib/cucloud/ec2_utils.rb', line 43

def stop_instance(instance)
  @ec2.stop_instances(instance_ids: [instance])
end

#stop_instances_by_tag(tag_name, tag_value) ⇒ Object

stop instances based on a tag name and value

Parameters:

  • tag_name (string) —

    name of tag

  • tag_value (string) —

    the value of the tag



108
109
110
111
112
 
# File 'lib/cucloud/ec2_utils.rb', line 108

def stop_instances_by_tag(tag_name, tag_value)
  get_instances_by_tag(tag_name, tag_value).reservations[0].instances.each do |i|
    @ec2.stop_instances(instance_ids: [i.instance_id])
  end
end

#terminate_instance(instance) ⇒ Object

Terminate ec2 instance for a specific instance number.



65
66
67
68
 
# File 'lib/cucloud/ec2_utils.rb', line 65

def terminate_instance(instance)
  i = get_instance(instance)
  i.terminate
end

#volumes_with_snapshot_within_last_days(days = 5) ⇒ Array

Find volumes that have a recent snapshot

Parameters:

  • days (Integer) (defaults to: 5) —

    defaults to 5

Returns:

  • (Array) —

    list of volume ids that have recent snapshots



227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/cucloud/ec2_utils.rb', line 227

def volumes_with_snapshot_within_last_days(days = 5)
  volumes_backed_up_recently = {}

  snapshots = @ec2.describe_snapshots(owner_ids: ['self'], filters: [{ name: 'status', values: ['completed'] }])
  snapshots.snapshots.each do |snapshot|
    if snapshot.start_time > Time.now - (SECONDS_IN_A_DAY * days)
      volumes_backed_up_recently[snapshot.volume_id.to_s] = true
    end
  end
  volumes_backed_up_recently
end