Class: Puppet::Resource::Catalog

Inherits:
SimpleGraph show all
Extended by:
Indirector, Util::Pson
Includes:
Util::Tagging
Defined in:
lib/vendor/puppet/resource/catalog.rb

Overview

This class models a node catalog. It is the thing meant to be passed from server to client, and it contains all of the information in the catalog, including the resources and the relationships between them.

Defined Under Namespace

Classes: ActiveRecord, Compiler, DuplicateResourceError, Queue, Rest, StaticCompiler, StoreConfigs, Yaml

Constant Summary collapse

Default_label =

Impose our container information on another graph by using it to replace any container vertices X with a pair of verticies { admissible_X and completed_X } such that that

0) completed_X depends on admissible_X
1) contents of X each depend on admissible_X
2) completed_X depends on each on the contents of X
3) everything which depended on X depens on completed_X
4) admissible_X depends on everything X depended on
5) the containers and their edges must be removed

Note that this requires attention to the possible case of containers which contain or depend on other containers, but has the advantage that the number of new edges created scales linearly with the number of contained verticies regardless of how containers are related; alternatives such as replacing container-edges with content-edges scale as the product of the number of external dependences, which is to say geometrically in the case of nested / chained containers.

{ :callback => :refresh, :event => :ALL_EVENTS }

Constants included from Indirector

Indirector::BadNameRegexp

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Indirector

configure_routes, indirects

Methods included from Util::Pson

pson_create

Methods included from Util::Tagging

#tag, #tagged?, #tags, #tags=

Methods inherited from SimpleGraph

#add_edge, #add_relationship, #add_vertex, #adjacent, #dependencies, #dependents, #direct_dependencies_of, #direct_dependents_of, #directed?, #dotty, #downstream_from_vertex, #each_edge, #edge?, #edges, #edges_between, #find_cycles_in_graph, #instance_variable_get, #leaves, #matching_edges, #path_between, #paths_in_cycle, #remove_edge!, #remove_vertex!, #report_cycles_in_graph, #reversal, #size, #tarjan, #to_a, #to_dot, #to_dot_graph, #to_yaml_properties, #tree_from_vertex, #upstream_from_vertex, #vertex?, #vertices, #walk, #write_cycles_to_graph, #yaml_initialize

Constructor Details

#initialize(name = nil) ⇒ Catalog

Returns a new instance of Catalog.



262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 262

def initialize(name = nil)
  super()
  @name = name if name
  @classes = []
  @resource_table = {}
  @transient_resources = []
  @applying = false
  @relationship_graph = nil

  @host_config = true

  @aliases = {}

  if block_given?
    yield(self)
    finalize
  end
end

Instance Attribute Details

#client_version ⇒ Object

Some metadata to help us compile and generally respond to the current state.



45
46
47
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 45

def client_version
  @client_version
end

#from_cache ⇒ Object

Whether this catalog was retrieved from the cache, which affects whether it is written back out again.



42
43
44
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 42

def from_cache
  @from_cache
end

#host_config ⇒ Object

Whether this is a host catalog, which behaves very differently. In particular, reports are sent, graphs are made, and state is stored in the state database. If this is set incorrectly, then you often end up in infinite loops, because catalogs are used to make things that the host catalog needs.



38
39
40
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 38

def host_config
  @host_config
end

#name ⇒ Object

The host name this is a catalog for.



24
25
26
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 24

def name
  @name
end

#retrieval_duration ⇒ Object

How long this catalog took to retrieve. Used for reporting stats.



31
32
33
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 31

def retrieval_duration
  @retrieval_duration
end

#server_version ⇒ Object

Some metadata to help us compile and generally respond to the current state.



45
46
47
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 45

def server_version
  @server_version
end

#version ⇒ Object

The catalog version. Used for testing whether a catalog is up to date.



28
29
30
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 28

def version
  @version
end

Class Method Details

.edge_from_pson(result, edge) ⇒ Object 



476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 476

def self.edge_from_pson(result, edge)
  # If no type information was presented, we manually find
  # the class.
  edge = Puppet::Relationship.from_pson(edge) if edge.is_a?(Hash)
  unless source = result.resource(edge.source)
    raise ArgumentError, "Could not convert from pson: Could not find relationship source #{edge.source.inspect}"
  end
  edge.source = source

  unless target = result.resource(edge.target)
    raise ArgumentError, "Could not convert from pson: Could not find relationship target #{edge.target.inspect}"
  end
  edge.target = target

  result.add_edge(edge)
end

.from_pson(data) ⇒ Object 



444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 444

def self.from_pson(data)
  result = new(data['name'])

  if tags = data['tags']
    result.tag(*tags)
  end

  if version = data['version']
    result.version = version
  end

  if resources = data['resources']
    resources = PSON.parse(resources) if resources.is_a?(String)
    resources.each do |res|
      resource_from_pson(result, res)
    end
  end

  if edges = data['edges']
    edges = PSON.parse(edges) if edges.is_a?(String)
    edges.each do |edge|
      edge_from_pson(result, edge)
    end
  end

  if classes = data['classes']
    result.add_class(*classes)
  end

  result
end

.resource_from_pson(result, res) ⇒ Object 



493
494
495
496
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 493

def self.resource_from_pson(result, res)
  res = Puppet::Resource.from_pson(res) if res.is_a? Hash
  result.add_resource(res)
end

Instance Method Details

#add_class(*classes) ⇒ Object

Add classes to our class list.



48
49
50
51
52
53
54
55
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 48

def add_class(*classes)
  classes.each do |klass|
    @classes << klass
  end

  # Add the class names as tags, too.
  tag(*classes)
end

#add_resource(*resource) ⇒ Object

Add a resource to our graph and to our resource table. This is actually a relatively complicated method, because it handles multiple aspects of Catalog behaviour:

  • Add the resource to the resource table

  • Add the resource to the resource graph

  • Add the resource to the relationship graph

  • Add any aliases that make sense for the resource (e.g., name != title)

Raises:

  • (ArgumentError) 


69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 69

def add_resource(*resource)
  add_resource(*resource[0..-2]) if resource.length > 1
  resource = resource.pop
  raise ArgumentError, "Can only add objects that respond to :ref, not instances of #{resource.class}" unless resource.respond_to?(:ref)
  fail_on_duplicate_type_and_title(resource)
  title_key = title_key_for_ref(resource.ref)

  @transient_resources << resource if applying?
  @resource_table[title_key] = resource

  # If the name and title differ, set up an alias

  if resource.respond_to?(:name) and resource.respond_to?(:title) and resource.respond_to?(:isomorphic?) and resource.name != resource.title
    self.alias(resource, resource.uniqueness_key) if resource.isomorphic?
  end

  resource.catalog = self if resource.respond_to?(:catalog=)
  add_vertex(resource)
  @relationship_graph.add_vertex(resource) if @relationship_graph
end

#alias(resource, key) ⇒ Object

Create an alias for a resource.



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 91

def alias(resource, key)
  resource.ref =~ /^(.+)\[/
  class_name = $1 || resource.class.name

  newref = [class_name, key].flatten

  if key.is_a? String
    ref_string = "#{class_name}[#{key}]"
    return if ref_string == resource.ref
  end

  # LAK:NOTE It's important that we directly compare the references,
  # because sometimes an alias is created before the resource is
  # added to the catalog, so comparing inside the below if block
  # isn't sufficient.
  if existing = @resource_table[newref]
    return if existing == resource
    resource_declaration = " at #{resource.file}:#{resource.line}" if resource.file and resource.line
    existing_declaration = " at #{existing.file}:#{existing.line}" if existing.file and existing.line
    msg = "Cannot alias #{resource.ref} to #{key.inspect}#{resource_declaration}; resource #{newref.inspect} already declared#{existing_declaration}"
    raise ArgumentError, msg
  end
  @resource_table[newref] = resource
  @aliases[resource.ref] ||= []
  @aliases[resource.ref] << newref
end

#apply(options = {}) ⇒ Object

Apply our catalog to the local host. Valid options are:

:tags - set the tags that restrict what resources run
    during the transaction
:ignoreschedules - tell the transaction to ignore schedules
    when determining the resources to run



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 124

def apply(options = {})
  @applying = true

  Puppet::Util::Storage.load if host_config?

  transaction = Puppet::Transaction.new(self, options[:report])
  register_report = options[:report].nil?

  transaction.tags = options[:tags] if options[:tags]
  transaction.ignoreschedules = true if options[:ignoreschedules]
  transaction.for_network_device = options[:network_device]

  transaction.add_times :config_retrieval => self.retrieval_duration || 0

  begin
    Puppet::Util::Log.newdestination(transaction.report) if register_report
    begin
      transaction.evaluate
    ensure
      Puppet::Util::Log.close(transaction.report) if register_report
    end
  rescue Puppet::Error => detail
    puts detail.backtrace if Puppet[:trace]
    Puppet.err "Could not apply complete catalog: #{detail}"
  rescue => detail
    puts detail.backtrace if Puppet[:trace]
    Puppet.err "Got an uncaught exception of type #{detail.class}: #{detail}"
  ensure
    # Don't try to store state unless we're a host config
    # too recursive.
    Puppet::Util::Storage.store if host_config?
  end

  yield transaction if block_given?

  return transaction
ensure
  @applying = false
end

#applying? ⇒ Boolean

Are we in the middle of applying the catalog?

Returns:

  • (Boolean) 


165
166
167
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 165

def applying?
  @applying
end

#classes ⇒ Object 



181
182
183
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 181

def classes
  @classes.dup
end

#clear(remove_resources = true) ⇒ Object 



169
170
171
172
173
174
175
176
177
178
179
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 169

def clear(remove_resources = true)
  super()
  # We have to do this so that the resources clean themselves up.
  @resource_table.values.each { |resource| resource.remove } if remove_resources
  @resource_table.clear

  if @relationship_graph
    @relationship_graph.clear
    @relationship_graph = nil
  end
end

#create_resource(type, options) ⇒ Object

Create a new resource and register it in the catalog.



186
187
188
189
190
191
192
193
194
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 186

def create_resource(type, options)
  unless klass = Puppet::Type.type(type)
    raise ArgumentError, "Unknown resource type #{type}"
  end
  return unless resource = klass.new(options)

  add_resource(resource)
  resource
end

#extract ⇒ Object

Turn our catalog graph into an old-style tree of TransObjects and TransBuckets. LAK:NOTE(20081211): This is a pre-0.25 backward compatibility method. It can be removed as soon as xmlrpc is killed.



199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 199

def extract
  top = nil
  current = nil
  buckets = {}

  unless main = resource(:stage, "main")
    raise Puppet::DevError, "Could not find 'main' stage; cannot generate catalog"
  end

  if stages = vertices.find_all { |v| v.type == "Stage" and v.title != "main" } and ! stages.empty?
    Puppet.warning "Stages are not supported by 0.24.x client; stage(s) #{stages.collect { |s| s.to_s }.join(', ') } will be ignored"
  end

  bucket = nil
  walk(main, :out) do |source, target|
    # The sources are always non-builtins.
    unless tmp = buckets[source.to_s]
      if tmp = buckets[source.to_s] = source.to_trans
        bucket = tmp
      else
        # This is because virtual resources return nil.  If a virtual
        # container resource contains realized resources, we still need to get
        # to them.  So, we keep a reference to the last valid bucket
        # we returned and use that if the container resource is virtual.
      end
    end
    bucket = tmp || bucket
    if child = target.to_trans
      raise "No bucket created for #{source}" unless bucket
      bucket.push child

      # It's important that we keep a reference to any TransBuckets we've created, so
      # we don't create multiple buckets for children.
      buckets[target.to_s] = child unless target.builtin?
    end
  end

  # Retrieve the bucket for the top-level scope and set the appropriate metadata.
  unless result = buckets[main.to_s]
    # This only happens when the catalog is entirely empty.
    result = buckets[main.to_s] = main.to_trans
  end

  result.classes = classes

  # Clear the cache to encourage the GC
  buckets.clear
  result
end

#filter(&block) ⇒ Object

filter out the catalog, applying block to each resource. If the block result is false, the resource will be kept otherwise it will be skipped



533
534
535
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 533

def filter(&block)
  to_catalog :to_resource, &block
end

#finalize ⇒ Object

Make sure all of our resources are “finished”.



250
251
252
253
254
255
256
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 250

def finalize
  make_default_resources

  @resource_table.values.each { |resource| resource.finish }

  write_graph(:resources)
end

#host_config? ⇒ Boolean

Returns:

  • (Boolean) 


258
259
260
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 258

def host_config?
  host_config
end

#make_default_resources ⇒ Object

Make the default objects necessary for function.



282
283
284
285
286
287
288
289
290
291
292
293
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 282

def make_default_resources
  # We have to add the resources to the catalog, or else they won't get cleaned up after
  # the transaction.

  # First create the default scheduling objects
  Puppet::Type.type(:schedule).mkdefaultschedules.each { |res| add_resource(res) unless resource(res.ref) }

  # And filebuckets
  if bucket = Puppet::Type.type(:filebucket).mkdefaultbucket
    add_resource(bucket) unless resource(bucket.ref)
  end
end

#relationship_graph ⇒ Object

Create a graph of all of the relationships in our catalog.



296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 296

def relationship_graph
  unless @relationship_graph
    # It's important that we assign the graph immediately, because
    # the debug messages below use the relationships in the
    # relationship graph to determine the path to the resources
    # spitting out the messages.  If this is not set,
    # then we get into an infinite loop.
    @relationship_graph = Puppet::SimpleGraph.new

    # First create the dependency graph
    self.vertices.each do |vertex|
      @relationship_graph.add_vertex vertex
      vertex.builddepends.each do |edge|
        @relationship_graph.add_edge(edge)
      end
    end

    # Lastly, add in any autorequires
    @relationship_graph.vertices.each do |vertex|
      vertex.autorequire(self).each do |edge|
        unless @relationship_graph.edge?(edge.source, edge.target) # don't let automatic relationships conflict with manual ones.
          unless @relationship_graph.edge?(edge.target, edge.source)
            vertex.debug "Autorequiring #{edge.source}"
            @relationship_graph.add_edge(edge)
          else
            vertex.debug "Skipping automatic relationship with #{(edge.source == vertex ? edge.target : edge.source)}"
          end
        end
      end
    end
    @relationship_graph.write_graph(:relationships) if host_config?

    # Then splice in the container information
    splice!(@relationship_graph)

    @relationship_graph.write_graph(:expanded_relationships) if host_config?
  end
  @relationship_graph
end

#remove_resource(*resources) ⇒ Object

Remove the resource from our catalog. Notice that we also call ‘remove’ on the resource, at least until resource classes no longer maintain references to the resource instances.



401
402
403
404
405
406
407
408
409
410
411
412
413
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 401

def remove_resource(*resources)
  resources.each do |resource|
    title_key = title_key_for_ref(resource.ref)
    @resource_table.delete(title_key)
    if aliases = @aliases[resource.ref]
      aliases.each { |res_alias| @resource_table.delete(res_alias) }
      @aliases.delete(resource.ref)
    end
    remove_vertex!(resource) if vertex?(resource)
    @relationship_graph.remove_vertex!(resource) if @relationship_graph and @relationship_graph.vertex?(resource)
    resource.remove
  end
end

#resource(type, title = nil) ⇒ Object

Look a resource up by its reference (e.g., File).



416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 416

def resource(type, title = nil)
  # Always create a resource reference, so that it always canonizes how we
  # are referring to them.
  if title
    res = Puppet::Resource.new(type, title)
  else
    # If they didn't provide a title, then we expect the first
    # argument to be of the form 'Class[name]', which our
    # Reference class canonizes for us.
    res = Puppet::Resource.new(nil, type)
  end
  title_key      = [res.type, res.title.to_s]
  uniqueness_key = [res.type, res.uniqueness_key].flatten
  @resource_table[title_key] || @resource_table[uniqueness_key]
end

#resource_keys ⇒ Object 



436
437
438
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 436

def resource_keys
  @resource_table.keys
end

#resource_refs ⇒ Object 



432
433
434
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 432

def resource_refs
  resource_keys.collect{ |type, name| name.is_a?( String ) ? "#{type}[#{name}]" : nil}.compact
end

#resources ⇒ Object 



440
441
442
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 440

def resources
  @resource_table.values.uniq
end

#splice!(other) ⇒ Object 



356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 356

def splice!(other)
  stage_class      = Puppet::Type.type(:stage)
  whit_class       = Puppet::Type.type(:whit)
  component_class  = Puppet::Type.type(:component)
  containers = vertices.find_all { |v| (v.is_a?(component_class) or v.is_a?(stage_class)) and vertex?(v) }
  #
  # These two hashes comprise the aforementioned attention to the possible
  #   case of containers that contain / depend on other containers; they map
  #   containers to their sentinels but pass other verticies through.  Thus we
  #   can "do the right thing" for references to other verticies that may or
  #   may not be containers.
  #
  admissible = Hash.new { |h,k| k }
  completed  = Hash.new { |h,k| k }
  containers.each { |x|
    admissible[x] = whit_class.new(:name => "admissible_#{x.ref}", :catalog => self)
    completed[x]  = whit_class.new(:name => "completed_#{x.ref}",  :catalog => self)
  }
  #
  # Implement the six requierments listed above
  #
  containers.each { |x|
    contents = adjacent(x, :direction => :out)
    other.add_edge(admissible[x],completed[x]) if contents.empty? # (0)
    contents.each { |v|
      other.add_edge(admissible[x],admissible[v],Default_label) # (1)
      other.add_edge(completed[v], completed[x], Default_label) # (2)
    }
    # (3) & (5)
    other.adjacent(x,:direction => :in,:type => :edges).each { |e|
      other.add_edge(completed[e.source],admissible[x],e.label)
      other.remove_edge! e
    }
    # (4) & (5)
    other.adjacent(x,:direction => :out,:type => :edges).each { |e|
      other.add_edge(completed[x],admissible[e.target],e.label)
      other.remove_edge! e
    }
  }
  containers.each { |x| other.remove_vertex! x } # (5)
end

#title_key_for_ref(ref) ⇒ Object 



57
58
59
60
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 57

def title_key_for_ref( ref )
  ref =~ /^([-\w:]+)\[(.*)\]$/m
  [$1, $2]
end

#to_pson(*args) ⇒ Object 



516
517
518
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 516

def to_pson(*args)
  to_pson_data_hash.to_pson(*args)
end

#to_pson_data_hash ⇒ Object 



499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 499

def to_pson_data_hash
  {
    'document_type' => 'Catalog',
    'data'       => {
      'tags'      => tags,
      'name'      => name,
      'version'   => version,
      'resources' => vertices.collect { |v| v.to_pson_data_hash },
      'edges'     => edges.   collect { |e| e.to_pson_data_hash },
      'classes'   => classes
      },
    'metadata' => {
      'api_version' => 1
      }
  }
end

#to_ral ⇒ Object

Convert our catalog into a RAL catalog.



521
522
523
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 521

def to_ral
  to_catalog :to_ral
end

#to_resource ⇒ Object

Convert our catalog into a catalog of Puppet::Resource instances.



526
527
528
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 526

def to_resource
  to_catalog :to_resource
end

#write_class_file ⇒ Object

Store the classes in the classfile.



538
539
540
541
542
543
544
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 538

def write_class_file
  ::File.open(Puppet[:classfile], "w") do |f|
    f.puts classes.join("\n")
  end
rescue => detail
  Puppet.err "Could not create class file #{Puppet[:classfile]}: #{detail}"
end

#write_graph(name) ⇒ Object

Produce the graph files if requested.



564
565
566
567
568
569
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 564

def write_graph(name)
  # We only want to graph the main host catalog.
  return unless host_config?

  super
end

#write_resource_file ⇒ Object

Store the list of resources we manage



547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/vendor/puppet/resource/catalog.rb', line 547

def write_resource_file
  ::File.open(Puppet[:resourcefile], "w") do |f|
    to_print = resources.map do |resource|
      next unless resource.managed?
      if resource.name_var
        "#{resource.type}[#{resource[resource.name_var]}]"
      else
        "#{resource.ref.downcase}"
      end
    end.compact
    f.puts to_print.join("\n")
  end
rescue => detail
  Puppet.err "Could not create resource file #{Puppet[:resourcefile]}: #{detail}"
end