Class: Bosh::Director::DeploymentPlan::Assembler
- Includes:
- Bosh::Director::DnsHelper, IpUtil, LockHelper
- Defined in:
- lib/bosh/director/deployment_plan/assembler.rb
Overview
DeploymentPlan::Assembler is used to populate deployment plan with information about existing deployment and information from director DB
Constant Summary
Constants included from Bosh::Director::DnsHelper
Bosh::Director::DnsHelper::SOA, Bosh::Director::DnsHelper::TTL_4H, Bosh::Director::DnsHelper::TTL_5M
Instance Method Summary collapse
-
#bind_configuration ⇒ void
Calculates configuration checksums for all jobs in this deployment plan.
-
#bind_deployment ⇒ void
Binds deployment DB record to a plan.
- #bind_dns ⇒ Object
-
#bind_existing_deployment ⇒ void
Binds information about existing deployment to a plan.
-
#bind_existing_vm(vm, lock) ⇒ Object
Queries agent for VM state and updates deployment plan accordingly.
-
#bind_idle_vm(vm, resource_pool, state, reservations) ⇒ Object
Binds idle VM to a resource pool with a proper network reservation.
- #bind_instance(instance_model, state, reservations) ⇒ Object
- #bind_instance_networks ⇒ Object
- #bind_instance_vm(instance) ⇒ Object
- #bind_instance_vms ⇒ Object
-
#bind_properties ⇒ void
Binds properties for all templates in the deployment.
-
#bind_releases ⇒ void
Binds release DB record(s) to a plan.
-
#bind_resource_pools ⇒ void
Takes a look at the current state of all resource pools in the deployment and schedules adding any new VMs if needed.
-
#bind_stemcells ⇒ void
Binds stemcell model for each stemcell spec in each resource pool in the deployment plan.
-
#bind_templates ⇒ void
Binds template models for each release spec in the deployment plan.
-
#bind_unallocated_vms ⇒ void
Looks at every job instance in the deployment plan and binds it to the instance database model (idle VM is also created in the appropriate resource pool if necessary).
- #delete_unneeded_instances ⇒ Object
- #delete_unneeded_vms ⇒ Object
- #get_network_reservations(state) ⇒ Object
- #get_state(vm) ⇒ Object
-
#initialize(deployment_plan) ⇒ Assembler
constructor
A new instance of Assembler.
- #migrate_legacy_state(vm, state) ⇒ Object
- #verify_state(vm, state) ⇒ Object
Methods included from IpUtil
#each_ip, #format_ip, #ip_to_i, #ip_to_netaddr, #process_range
Methods included from LockHelper
#with_compile_lock, #with_deployment_lock, #with_release_lock, #with_release_locks, #with_stemcell_lock
Methods included from Bosh::Director::DnsHelper
#add_default_dns_server, #canonical, #default_dns_server, #delete_dns_records, #delete_empty_domain, #dns_domain_name, #dns_ns_record, #dns_servers, #invalid_dns, #reverse_domain, #reverse_host, #update_dns_a_record, #update_dns_ptr_record
Constructor Details
#initialize(deployment_plan) ⇒ Assembler
Returns a new instance of Assembler.
10 11 12 13 14 15 16 17 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 10 def initialize(deployment_plan) @deployment_plan = deployment_plan @cloud = Config.cloud @logger = Config.logger @event_log = Config.event_log @stemcell_manager = Api::StemcellManager.new @blobstore = App.instance.blobstores.blobstore end |
Instance Method Details
#bind_configuration ⇒ void
This method returns an undefined value.
Calculates configuration checksums for all jobs in this deployment plan
330 331 332 333 334 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 330 def bind_configuration @deployment_plan.jobs.each do |job| JobRenderer.new(job).render_job_instances(@blobstore) end end |
#bind_deployment ⇒ void
This method returns an undefined value.
Binds deployment DB record to a plan
21 22 23 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 21 def bind_deployment @deployment_plan.bind_model end |
#bind_dns ⇒ Object
336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 336 def bind_dns domain = Models::Dns::Domain.find_or_create(:name => dns_domain_name, :type => 'NATIVE') @deployment_plan.dns_domain = domain soa_record = Models::Dns::Record.find_or_create(:domain_id => domain.id, :name => dns_domain_name, :type => 'SOA') soa_record.content = SOA soa_record.ttl = 300 soa_record.save # add NS record Models::Dns::Record.find_or_create(:domain_id => domain.id, :name => dns_domain_name, :type =>'NS', :ttl => TTL_4H, :content => dns_ns_record) # add A record for name server Models::Dns::Record.find_or_create(:domain_id => domain.id, :name => dns_ns_record, :type =>'A', :ttl => TTL_4H, :content => Config.dns['address']) end |
#bind_existing_deployment ⇒ void
This method returns an undefined value.
Binds information about existing deployment to a plan
37 38 39 40 41 42 43 44 45 46 47 48 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 37 def bind_existing_deployment lock = Mutex.new ThreadPool.new(:max_threads => Config.max_threads).wrap do |pool| @deployment_plan.vms.each do |vm| pool.process do with_thread_name("bind_existing_deployment(#{vm.agent_id})") do bind_existing_vm(vm, lock) end end end end end |
#bind_existing_vm(vm, lock) ⇒ Object
Queries agent for VM state and updates deployment plan accordingly
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 53 def bind_existing_vm(vm, lock) state = get_state(vm) lock.synchronize do @logger.debug('Processing network reservations') reservations = get_network_reservations(state) instance = vm.instance if instance bind_instance(instance, state, reservations) else @logger.debug('Binding resource pool VM') resource_pool = @deployment_plan.resource_pool( state['resource_pool']['name']) if resource_pool bind_idle_vm(vm, resource_pool, state, reservations) else @logger.debug("Resource pool doesn't exist, marking for deletion") @deployment_plan.delete_vm(vm) end end @logger.debug('Finished binding VM') end end |
#bind_idle_vm(vm, resource_pool, state, reservations) ⇒ Object
Binds idle VM to a resource pool with a proper network reservation
82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 82 def bind_idle_vm(vm, resource_pool, state, reservations) @logger.debug('Adding to resource pool') idle_vm = resource_pool.add_idle_vm idle_vm.vm = vm idle_vm.current_state = state reservation = reservations[resource_pool.network.name] if reservation if reservation.static? @logger.debug('Releasing static network reservation for ' + "resource pool VM `#{vm.cid}'") resource_pool.network.release(reservation) else idle_vm.use_reservation(reservation) end else @logger.debug("No network reservation for VM `#{vm.cid}'") end end |
#bind_instance(instance_model, state, reservations) ⇒ Object
105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 105 def bind_instance(instance_model, state, reservations) @logger.debug('Binding instance VM') # Update instance, if we are renaming a job. if @deployment_plan.rename_in_progress? old_name = @deployment_plan.job_rename['old_name'] new_name = @deployment_plan.job_rename['new_name'] if instance_model.job == old_name @logger.info("Renaming `#{old_name}' to `#{new_name}'") instance_model.update(:job => new_name) end end # Does the job instance exist in the new deployment? if (job = @deployment_plan.job(instance_model.job)) && (instance = job.instance(instance_model.index)) @logger.debug('Found job and instance spec') instance.use_model(instance_model) instance.current_state = state @logger.debug('Copying network reservations') instance.take_network_reservations(reservations) @logger.debug('Copying resource pool reservation') job.resource_pool.mark_active_vm else @logger.debug('Job/instance not found, marking for deletion') @deployment_plan.delete_instance(instance_model) end end |
#bind_instance_networks ⇒ Object
279 280 281 282 283 284 285 286 287 288 289 290 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 279 def bind_instance_networks @deployment_plan.jobs.each do |job| job.instances.each do |instance| instance.network_reservations.each do |name, reservation| unless reservation.reserved? network = @deployment_plan.network(name) network.reserve!(reservation, "`#{job.name}/#{instance.index}'") end end end end end |
#bind_instance_vm(instance) ⇒ Object
387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 387 def bind_instance_vm(instance) @event_log.track("#{instance.job.name}/#{instance.index}") do idle_vm = instance.idle_vm # Apply the assignment to the VM agent = AgentClient.with_defaults(idle_vm.vm.agent_id) state = idle_vm.current_state state['job'] = instance.job.spec state['index'] = instance.index agent.apply(state) # Our assumption here is that director database access # is much less likely to fail than VM agent communication # so we only update database after we see a successful agent apply. # If database update fails subsequent deploy will try to # assign a new VM to this instance which is ok. idle_vm.vm.db.transaction do idle_vm.vm.update(:apply_spec => state) instance.model.update(:vm => idle_vm.vm) end instance.current_state = state end end |
#bind_instance_vms ⇒ Object
360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 360 def bind_instance_vms unbound_instances = [] @deployment_plan.jobs.each do |job| job.instances.each do |instance| # Don't allocate resource pool VMs to instances in detached state next if instance.state == 'detached' # Skip bound instances next if instance.model.vm unbound_instances << instance end end return if unbound_instances.empty? @event_log.begin_stage('Binding instance VMs', unbound_instances.size) ThreadPool.new(:max_threads => Config.max_threads).wrap do |pool| unbound_instances.each do |instance| pool.process do bind_instance_vm(instance) end end end end |
#bind_properties ⇒ void
This method returns an undefined value.
Binds properties for all templates in the deployment
306 307 308 309 310 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 306 def bind_properties @deployment_plan.jobs.each do |job| job.bind_properties end end |
#bind_releases ⇒ void
This method returns an undefined value.
Binds release DB record(s) to a plan
27 28 29 30 31 32 33 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 27 def bind_releases with_release_locks(@deployment_plan) do @deployment_plan.releases.each do |release| release.bind_model end end end |
#bind_resource_pools ⇒ void
This method returns an undefined value.
Takes a look at the current state of all resource pools in the deployment and schedules adding any new VMs if needed. VMs are NOT created at this stage, only data structures are being allocated. ResourcePoolUpdater will later perform actual changes based on this data.
258 259 260 261 262 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 258 def bind_resource_pools @deployment_plan.resource_pools.each do |resource_pool| resource_pool.process_idle_vms end end |
#bind_stemcells ⇒ void
This method returns an undefined value.
Binds stemcell model for each stemcell spec in each resource pool in the deployment plan
315 316 317 318 319 320 321 322 323 324 325 326 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 315 def bind_stemcells @deployment_plan.resource_pools.each do |resource_pool| stemcell = resource_pool.stemcell if stemcell.nil? raise DirectorError, "Stemcell not bound for resource pool `#{resource_pool.name}'" end stemcell.bind_model end end |
#bind_templates ⇒ void
This method returns an undefined value.
Binds template models for each release spec in the deployment plan
294 295 296 297 298 299 300 301 302 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 294 def bind_templates @deployment_plan.releases.each do |release| release.bind_templates end @deployment_plan.jobs.each do |job| job.validate_package_names_do_not_collide! end end |
#bind_unallocated_vms ⇒ void
This method returns an undefined value.
Looks at every job instance in the deployment plan and binds it to the instance database model (idle VM is also created in the appropriate resource pool if necessary)
268 269 270 271 272 273 274 275 276 277 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 268 def bind_unallocated_vms @deployment_plan.jobs.each do |job| job.instances.each do |instance| instance.bind_unallocated_vm # Now that we know every VM has been allocated and instance models are # bound, we can sync the state. instance.sync_state_with_db end end end |
#delete_unneeded_instances ⇒ Object
432 433 434 435 436 437 438 439 440 441 442 443 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 432 def delete_unneeded_instances unneeded_instances = @deployment_plan.unneeded_instances if unneeded_instances.empty? @logger.info('No unneeded instances to delete') return end event_log_stage = @event_log.begin_stage('Deleting unneeded instances', unneeded_instances.size) instance_deleter = InstanceDeleter.new(@deployment_plan) instance_deleter.delete_instances(unneeded_instances, event_log_stage) @logger.info('Deleted no longer needed instances') end |
#delete_unneeded_vms ⇒ Object
411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 411 def delete_unneeded_vms unneeded_vms = @deployment_plan.unneeded_vms if unneeded_vms.empty? @logger.info('No unneeded vms to delete') return end @event_log.begin_stage('Deleting unneeded VMs', unneeded_vms.size) ThreadPool.new(:max_threads => Config.max_threads).wrap do |pool| unneeded_vms.each do |vm| pool.process do @event_log.track(vm.cid) do @logger.info("Delete unneeded VM #{vm.cid}") @cloud.delete_vm(vm.cid) vm.destroy end end end end end |
#get_network_reservations(state) ⇒ Object
138 139 140 141 142 143 144 145 146 147 148 149 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 138 def get_network_reservations(state) reservations = {} state['networks'].each do |name, network_config| network = @deployment_plan.network(name) if network reservation = NetworkReservation.new(:ip => network_config['ip']) network.reserve(reservation) reservations[name] = reservation if reservation.reserved? end end reservations end |
#get_state(vm) ⇒ Object
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 151 def get_state(vm) @logger.debug("Requesting current VM state for: #{vm.agent_id}") agent = AgentClient.with_defaults(vm.agent_id) state = agent.get_state @logger.debug("Received VM state: #{state.pretty_inspect}") verify_state(vm, state) @logger.debug('Verified VM state') migrate_legacy_state(vm, state) state.delete('release') if state.include?('job') state['job'].delete('release') end state end |
#migrate_legacy_state(vm, state) ⇒ Object
232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 232 def migrate_legacy_state(vm, state) # Persisting apply spec for VMs that were introduced before we started # persisting it on apply itself (this is for cloudcheck purposes only) if vm.apply_spec.nil? # The assumption is that apply_spec <=> VM state vm.update(:apply_spec => state) end instance = vm.instance if instance disk_size = state['persistent_disk'].to_i persistent_disk = instance.persistent_disk # This is to support legacy deployments where we did not have # the disk_size specified. if disk_size != 0 && persistent_disk && persistent_disk.size == 0 persistent_disk.update(:size => disk_size) end end end |
#verify_state(vm, state) ⇒ Object
168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 |
# File 'lib/bosh/director/deployment_plan/assembler.rb', line 168 def verify_state(vm, state) instance = vm.instance if instance && instance.deployment_id != vm.deployment_id # Both VM and instance should reference same deployment raise VmInstanceOutOfSync, "VM `#{vm.cid}' and instance " + "`#{instance.job}/#{instance.index}' " + "don't belong to the same deployment" end unless state.kind_of?(Hash) @logger.error("Invalid state for `#{vm.cid}': #{state.pretty_inspect}") raise AgentInvalidStateFormat, "VM `#{vm.cid}' returns invalid state: " + "expected Hash, got #{state.class}" end actual_deployment_name = state['deployment'] expected_deployment_name = @deployment_plan.name if actual_deployment_name != expected_deployment_name raise AgentWrongDeployment, "VM `#{vm.cid}' is out of sync: " + 'expected to be a part of deployment ' + "`#{expected_deployment_name}' " + 'but is actually a part of deployment ' + "`#{actual_deployment_name}'" end actual_job = state['job'].is_a?(Hash) ? state['job']['name'] : nil actual_index = state['index'] if instance.nil? && !actual_job.nil? raise AgentUnexpectedJob, "VM `#{vm.cid}' is out of sync: " + "it reports itself as `#{actual_job}/#{actual_index}' but " + 'there is no instance reference in DB' end if instance && (instance.job != actual_job || instance.index != actual_index) # Check if we are resuming a previously unfinished rename if actual_job == @deployment_plan.job_rename['old_name'] && instance.job == @deployment_plan.job_rename['new_name'] && instance.index == actual_index # Rename already happened in the DB but then something happened # and agent has never been updated. unless @deployment_plan.job_rename['force'] raise AgentRenameInProgress, "Found a job `#{actual_job}' that seems to be " + "in the middle of a rename to `#{instance.job}'. " + "Run 'rename' again with '--force' to proceed." end else raise AgentJobMismatch, "VM `#{vm.cid}' is out of sync: " + "it reports itself as `#{actual_job}/#{actual_index}' but " + "according to DB it is `#{instance.job}/#{instance.index}'" end end end |