Class: Ufo::Ship
- Inherits:
-
Object
- Object
- Ufo::Ship
- Includes:
- AwsServices, Defaults, Util
- Defined in:
- lib/ufo/ship.rb
Instance Method Summary collapse
-
#add_load_balancer!(container, options) ⇒ Object
Only support Application Load Balancer Think there is an AWS bug that complains about not having the LB name but you cannot pass both a LB Name and a Target Group.
- #cluster_arn ⇒ Object
-
#container_info(task_definition) ⇒ Object
assume only 1 container_definition assume only 1 port mapping in that container_defintion.
-
#create_service ⇒ Object
$ aws ecs create-service –generate-cli-skeleton { “cluster”: “”, “serviceName”: “”, “taskDefinition”: “”, “desiredCount”: 0, “loadBalancers”: [ { “targetGroupArn”: “”, “containerName”: “”, “containerPort”: 0 } ], “role”: “”, “clientToken”: “”, “deploymentConfiguration”: { “maximumPercent”: 0, “minimumHealthyPercent”: 0 } }.
-
#create_service_prompt(container) ⇒ Object
Returns the target_group.
-
#deploy ⇒ Object
If it looks like a regexp is passed in then it’ll only update the services This is because regpex cannot be used to determined a list of service_names.
-
#deployment_complete(deployed_service) ⇒ Object
aws ecs describe-services –services hi-web-prod –cluster prod-hi Passing in the service because we need to capture the deployed task_definition that was actually deployed.
- #ecs_clusters ⇒ Object
- #ensure_cluster_exist ⇒ Object
-
#find_all_ecs_services ⇒ Object
find all services on a cluster yields ECS::Service object.
- #find_ecs_service ⇒ Object
-
#find_updated_service(service) ⇒ Object
used for polling must pass in a service and cannot use @service for the case of multi_services mode.
-
#initialize(service, task_definition, options = {}) ⇒ Ship
constructor
A new instance of Ship.
- #old_task?(deployed_task_definition_arn, task_definition_arn) ⇒ Boolean
-
#process_single_service ⇒ Object
A single service name shouold had been passed and the service automatically gets created if it does not exist.
- #service_arns ⇒ Object
- #service_tasks(cluster, service) ⇒ Object
-
#stop_old_task(deployed_service) ⇒ Object
aws ecs list-tasks –cluster prod-hi –service-name gr-web-prod aws ecs describe-tasks –tasks arn:aws:ecs:us-east-1:467446852200:task/09038fd2-f989-4903-a8c6-1bc41761f93f –cluster prod-hi.
- #stop_old_tasks(services) ⇒ Object
- #task_name(task_definition) ⇒ Object
- #task_version(task_definition) ⇒ Object
-
#update_service(ecs_service) ⇒ Object
$ aws ecs update-service –generate-cli-skeleton { “cluster”: “”, “service”: “”, “taskDefinition”: “”, “desiredCount”: 0, “deploymentConfiguration”: { “maximumPercent”: 0, “minimumHealthyPercent”: 0 } } Only thing we want to change is the task-definition.
- #validate_target_group(arn) ⇒ Object
- #wait_for_all_deployments(deployed_services) ⇒ Object
-
#wait_for_deployment(deployed_service, quiet = false) ⇒ Object
service is the returned object from aws-sdk not the @service which is just a String.
Methods included from Util
Methods included from AwsServices
Methods included from Defaults
#default_cluster, #default_desired_count, #default_maximum_percent, #default_minimum_healthy_percent, #new_service_settings, #settings
Constructor Details
#initialize(service, task_definition, options = {}) ⇒ Ship
Returns a new instance of Ship.
29 30 31 32 33 34 35 36 37 38 |
# File 'lib/ufo/ship.rb', line 29 def initialize(service, task_definition, ={}) @service = service @task_definition = task_definition @options = @project_root = [:project_root] || '.' @target_group_prompt = @options[:target_group_prompt].nil? ? true : @options[:target_group_prompt] @cluster = @options[:cluster] || default_cluster @wait_for_deployment = @options[:wait].nil? ? true : @options[:wait] @stop_old_tasks = @options[:stop_old_tasks].nil? ? false : @options[:stop_old_tasks] end |
Instance Method Details
#add_load_balancer!(container, options) ⇒ Object
Only support Application Load Balancer Think there is an AWS bug that complains about not having the LB name but you cannot pass both a LB Name and a Target Group.
288 289 290 291 292 293 294 295 296 297 298 299 |
# File 'lib/ufo/ship.rb', line 288 def add_load_balancer!(container, ) .merge!( role: "ecsServiceRole", # assumption that we're using the ecsServiceRole load_balancers: [ { container_name: container[:name], container_port: container[:port], target_group_arn: @options[:target_group], } ] ) end |
#cluster_arn ⇒ Object
380 381 382 |
# File 'lib/ufo/ship.rb', line 380 def cluster_arn @cluster_arn ||= ecs_clusters.first.cluster_arn end |
#container_info(task_definition) ⇒ Object
assume only 1 container_definition assume only 1 port mapping in that container_defintion
339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 |
# File 'lib/ufo/ship.rb', line 339 def container_info(task_definition) task_definition_path = "ufo/output/#{task_definition}.json" task_definition_full_path = "#{@project_root}/#{task_definition_path}" unless File.exist?(task_definition_full_path) puts "ERROR: Unable to find the task definition at #{task_definition_path}.".colorize(:red) puts "Are you sure you have defined it in ufo/template_definitions.rb?".colorize(:red) exit end task_definition = JSON.load(IO.read(task_definition_full_path)) container_def = task_definition["containerDefinitions"].first mappings = container_def["portMappings"] if mappings map = mappings.first port = map["containerPort"] end { name: container_def["name"], port: port } end |
#create_service ⇒ Object
$ aws ecs create-service –generate-cli-skeleton {
"cluster": "",
"serviceName": "",
"taskDefinition": "",
"desiredCount": 0,
"loadBalancers": [
{
"targetGroupArn": "",
"containerName": "",
"containerPort": 0
}
],
"role": "",
"clientToken": "",
"deploymentConfiguration": {
"maximumPercent": 0,
"minimumHealthyPercent": 0
}
}
If the service needs to be created it will get created with some default settings. When does a normal deploy where an update happens only the only thing that ufo will update is the task_definition. The other settings should normally be updated with the ECS console. ‘ufo scale` will allow you to updated the desired_count from the CLI though.
229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 |
# File 'lib/ufo/ship.rb', line 229 def create_service container = container_info(@task_definition) target_group = create_service_prompt(container) = "#{@service} service created on #{@cluster} cluster" if @options[:noop] = "NOOP #{}" else = { cluster: @cluster, service_name: @service, desired_count: default_desired_count, deployment_configuration: { maximum_percent: default_maximum_percent, minimum_healthy_percent: default_minimum_healthy_percent }, task_definition: @task_definition } unless target_group.nil? || target_group.empty? add_load_balancer!(container, ) end response = ecs.create_service() service = response.service # must set service here since this might never be called if @wait_for_deployment is false end puts unless @options[:mute] service end |
#create_service_prompt(container) ⇒ Object
Returns the target_group. Will only allow an target_group and the service to use a load balancer if the container name is “web”.
304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 |
# File 'lib/ufo/ship.rb', line 304 def create_service_prompt(container) return if @options[:noop] return unless @target_group_prompt if container[:name] != "web" and @options[:target_group] puts "WARNING: A --target-group #{@options[:target_group]} was provided but it will not be used because this not a web container. Container name: #{container[:name].inspect}." end return unless container[:name] == 'web' return @options[:target_group] if @options[:target_group] puts "This service #{@service} does not yet exist in the #{@cluster} cluster. This deploy will create it." puts "Would you like this service to be associated with an Application Load Balancer?" puts "If yes, please provide the Application Load Balancer Target Group ARN." puts "If no, simply press enter." print "Target Group ARN: " arn = $stdin.gets.strip until arn == '' or validate_target_group(arn) puts "You have provided an invalid Application Load Balancer Target Group ARN: #{arn}." puts "It should be in the form: arn:aws:elasticloadbalancing:us-east-1:123456789:targetgroup/target-name/2378947392743" puts "Please try again or skip adding a Target Group by just pressing enter." print "Target Group ARN: " arn = $stdin.gets.strip end arn end |
#deploy ⇒ Object
If it looks like a regexp is passed in then it’ll only update the services This is because regpex cannot be used to determined a list of service_names.
Example:
No way to map: hi-.*-prod -> hi-web-prod hi-worker-prod hi-clock-prod
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# File 'lib/ufo/ship.rb', line 52 def deploy = "Shipping #{@service}..." unless @options[:mute] if @options[:noop] puts "NOOP: #{}" return else puts .green end end ensure_cluster_exist process_single_service puts "Software shipped!" unless @options[:mute] end |
#deployment_complete(deployed_service) ⇒ Object
aws ecs describe-services –services hi-web-prod –cluster prod-hi Passing in the service because we need to capture the deployed task_definition that was actually deployed. We use it to pull the describe_services until all the paramters we expect upon a completed deployment are updated.
188 189 190 191 192 193 194 195 196 197 198 199 200 201 |
# File 'lib/ufo/ship.rb', line 188 def deployment_complete(deployed_service) deployed_task_definition = deployed_service.task_definition # want the stale task_definition out of the wa service = find_updated_service(deployed_service) # polling deployment = service.deployments.first # Edge case when another deploy superseds this deploy in this case break out of this loop deployed_task_version = task_version(deployed_task_definition) current_task_version = task_version(service.task_definition) if current_task_version > deployed_task_version raise ShipmentOverridden.new("deployed_task_version was #{deployed_task_version} but task_version is now #{current_task_version}") end (deployment.task_definition == deployed_task_definition && deployment.desired_count == deployment.running_count) end |
#ecs_clusters ⇒ Object
397 398 399 |
# File 'lib/ufo/ship.rb', line 397 def ecs_clusters ecs.describe_clusters(clusters: [@cluster]).clusters end |
#ensure_cluster_exist ⇒ Object
384 385 386 387 388 389 390 391 392 393 394 395 |
# File 'lib/ufo/ship.rb', line 384 def ensure_cluster_exist cluster_exist = ecs_clusters.first unless cluster_exist = "#{@cluster} cluster created." if @options[:noop] = "NOOP #{}" else ecs.create_cluster(cluster_name: @cluster) end puts unless @options[:mute] end end |
#find_all_ecs_services ⇒ Object
find all services on a cluster yields ECS::Service object
366 367 368 369 370 371 372 373 374 |
# File 'lib/ufo/ship.rb', line 366 def find_all_ecs_services ecs_services = [] service_arns.each do |service_arn| ecs_service = ECS::Service.new(cluster_arn, service_arn) yield(ecs_service) if block_given? ecs_services << ecs_service end ecs_services end |
#find_ecs_service ⇒ Object
360 361 362 |
# File 'lib/ufo/ship.rb', line 360 def find_ecs_service find_all_ecs_services.find { |ecs_service| ecs_service.service_name == @service } end |
#find_updated_service(service) ⇒ Object
used for polling must pass in a service and cannot use @service for the case of multi_services mode
179 180 181 |
# File 'lib/ufo/ship.rb', line 179 def find_updated_service(service) ecs.describe_services(services: [service.service_name], cluster: @cluster).services.first end |
#old_task?(deployed_task_definition_arn, task_definition_arn) ⇒ Boolean
91 92 93 94 95 96 97 98 99 100 101 |
# File 'lib/ufo/ship.rb', line 91 def old_task?(deployed_task_definition_arn, task_definition_arn) puts "deployed_task_definition_arn: #{deployed_task_definition_arn.inspect}" puts "task_definition_arn: #{task_definition_arn.inspect}" deployed_version = deployed_task_definition_arn.split(':').last.to_i version = task_definition_arn.split(':').last.to_i puts "deployed_version #{deployed_version.inspect}" puts "version #{version.inspect}" is_old = version < deployed_version puts "is_old #{is_old.inspect}" is_old end |
#process_single_service ⇒ Object
A single service name shouold had been passed and the service automatically gets created if it does not exist.
71 72 73 74 75 76 77 78 79 80 81 82 83 |
# File 'lib/ufo/ship.rb', line 71 def process_single_service ecs_service = find_ecs_service deployed_service = if ecs_service # update all existing service update_service(ecs_service) else # create service on the first cluster create_service end wait_for_deployment(deployed_service) if @wait_for_deployment && !@options[:noop] stop_old_task(deployed_service) if @stop_old_tasks end |
#service_arns ⇒ Object
376 377 378 |
# File 'lib/ufo/ship.rb', line 376 def service_arns ecs.list_services(cluster: @cluster).service_arns end |
#service_tasks(cluster, service) ⇒ Object
85 86 87 88 89 |
# File 'lib/ufo/ship.rb', line 85 def service_tasks(cluster, service) all_task_arns = ecs.list_tasks(cluster: cluster, service_name: service).task_arns return [] if all_task_arns.empty? ecs.describe_tasks(cluster: cluster, tasks: all_task_arns).tasks end |
#stop_old_task(deployed_service) ⇒ Object
aws ecs list-tasks –cluster prod-hi –service-name gr-web-prod aws ecs describe-tasks –tasks arn:aws:ecs:us-east-1:467446852200:task/09038fd2-f989-4903-a8c6-1bc41761f93f –cluster prod-hi
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
# File 'lib/ufo/ship.rb', line 111 def stop_old_task(deployed_service) deployed_task_definition_arn = deployed_service.task_definition puts "deployed_task_definition_arn #{deployed_task_definition_arn.inspect}" # cannot use @serivce because of multiple mode all_tasks = service_tasks(@cluster, deployed_service.service_name) old_tasks = all_tasks.select do |task| old_task?(deployed_task_definition_arn, task.task_definition_arn) end reason = "Ufo #{Ufo::VERSION} has deployed new code and waited until the newer code is running." puts reason # Stopping old tasks after we have confirmed that the new task definition has the same # number of desired_count and running_count speeds up clean up and ensure that we # dont have any stale code being served. It seems to take a long time for the # ELB to drain the register container otherwise. This might cut off some requests but # providing this as an option that can be turned of beause I've seen deploys go way too # slow. puts "@options[:stop_old_tasks] #{@options[:stop_old_tasks].inspect}" puts "old_tasks.size #{old_tasks.size}" old_tasks.each do |task| puts "stopping task.task_definition_arn #{task.task_definition_arn.inspect}" ecs.stop_task(cluster: @cluster, task: task.task_arn, reason: reason) end if @options[:stop_old_tasks] end |
#stop_old_tasks(services) ⇒ Object
103 104 105 106 107 |
# File 'lib/ufo/ship.rb', line 103 def stop_old_tasks(services) services.each do |service| stop_old_task(service) end end |
#task_name(task_definition) ⇒ Object
401 402 403 404 405 406 |
# File 'lib/ufo/ship.rb', line 401 def task_name(task_definition) # "arn:aws:ecs:us-east-1:123456789:task-definition/hi-web-prod:72" # -> # "task-definition/hi-web-prod:72" task_definition.split('/').last end |
#task_version(task_definition) ⇒ Object
408 409 410 411 |
# File 'lib/ufo/ship.rb', line 408 def task_version(task_definition) # "task-definition/hi-web-prod:72" -> 72 task_name(task_definition).split(':').last.to_i end |
#update_service(ecs_service) ⇒ Object
$ aws ecs update-service –generate-cli-skeleton {
"cluster": "",
"service": "",
"taskDefinition": "",
"desiredCount": 0,
"deploymentConfiguration": {
"maximumPercent": 0,
"minimumHealthyPercent": 0
}
} Only thing we want to change is the task-definition
269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 |
# File 'lib/ufo/ship.rb', line 269 def update_service(ecs_service) = "#{ecs_service.service_name} service updated on #{ecs_service.cluster_name} cluster with task #{@task_definition}" if @options[:noop] = "NOOP #{}" else response = ecs.update_service( cluster: ecs_service.cluster_arn, # can use the cluster name also since it is unique service: ecs_service.service_arn, # can use the service name also since it is unique task_definition: @task_definition ) service = response.service # must set service here since this might never be called if @wait_for_deployment is false end puts unless @options[:mute] service end |
#validate_target_group(arn) ⇒ Object
330 331 332 333 334 335 |
# File 'lib/ufo/ship.rb', line 330 def validate_target_group(arn) elb.describe_target_groups(target_group_arns: [arn]) true rescue Aws::ElasticLoadBalancingV2::Errors::ValidationError false end |
#wait_for_all_deployments(deployed_services) ⇒ Object
158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 |
# File 'lib/ufo/ship.rb', line 158 def wait_for_all_deployments(deployed_services) start_time = Time.now threads = deployed_services.map do |deployed_service| Thread.new do # http://stackoverflow.com/questions/1383390/how-can-i-return-a-value-from-a-thread-in-ruby Thread.current[:output] = wait_for_deployment(deployed_service, quiet=true) end end threads.each { |t| t.join } total_took = Time.now - start_time puts "" puts "Shipments for all #{deployed_service.size} services took a total of #{pretty_time(total_took).green}." puts "Each deployment took:" threads.each do |t| service_name, took = t[:output] puts " #{service_name}: #{pretty_time(took)}" end end |
#wait_for_deployment(deployed_service, quiet = false) ⇒ Object
service is the returned object from aws-sdk not the @service which is just a String. Returns [service_name, time_took]
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
# File 'lib/ufo/ship.rb', line 139 def wait_for_deployment(deployed_service, quiet=false) start_time = Time.now deployed_task_name = task_name(deployed_service.task_definition) puts "Waiting for deployment of task definition #{deployed_task_name.green} to complete" unless quiet begin until deployment_complete(deployed_service) print '.' sleep 5 end rescue ShipmentOverridden => e puts "This deployed was overridden by another deploy" puts e. end puts '' unless quiet took = Time.now - start_time puts "Time waiting for ECS deployment: #{pretty_time(took).green}." unless quiet [deployed_service.service_name, took] end |