Class: Datadog::DI::ProbeManager Private

Inherits:
Object
  • Object
show all
Defined in:
lib/datadog/di/probe_manager.rb

Overview

This class is part of a private API. You should avoid using this class if possible, as it may be removed or be changed in the future.

Orchestrates probe lifecycle: installation, removal, and execution callbacks. Delegates probe storage to ProbeRepository.

Stores probes received from remote config (that we can parse, in other words, whose type/attributes we support), requests needed instrumentation for the probes via Instrumenter, and stores pending probes (those which haven't yet been instrumented successfully due to their targets not existing) and failed probes (where we are certain the target will not ever be loaded, or otherwise become valid).

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(settings, instrumenter, probe_notification_builder, probe_notifier_worker, logger, probe_repository, telemetry: nil) ⇒ ProbeManager

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of ProbeManager.



21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/datadog/di/probe_manager.rb', line 21

def initialize(settings, instrumenter, probe_notification_builder,
  probe_notifier_worker, logger, probe_repository, telemetry: nil)
  @settings = settings
  @instrumenter = instrumenter
  @probe_notification_builder = probe_notification_builder
  @probe_notifier_worker = probe_notifier_worker
  @logger = logger
  @telemetry = telemetry
  @probe_repository = probe_repository

  @definition_trace_point = TracePoint.new(:end) do |tp|
    install_pending_method_probes(tp.self)
  rescue Exception => exc # standard:disable Lint/RescueException
    Datadog::DI.reraise_if_fatal(exc)
    raise if settings.dynamic_instrumentation.internal.propagate_all_exceptions
    logger.debug { "di: unhandled exception in definition trace point: #{exc.class}: #{exc.message}" }
    telemetry&.report(exc, description: "Unhandled exception in definition trace point")
  end
end

Instance Attribute Details

#definition_trace_point ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Class/module definition trace point (:end type). Used to install hooks when the target classes/modules aren't yet defined when the hook request is received.



342
343
344
 
# File 'lib/datadog/di/probe_manager.rb', line 342

def definition_trace_point
  @definition_trace_point
end

#instrumenter ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



126
127
128
 
# File 'lib/datadog/di/probe_manager.rb', line 126

def instrumenter
  @instrumenter
end

#logger ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



41
42
43
 
# File 'lib/datadog/di/probe_manager.rb', line 41

def logger
  @logger
end

#probe_notification_builder ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



127
128
129
 
# File 'lib/datadog/di/probe_manager.rb', line 127

def probe_notification_builder
  @probe_notification_builder
end

#probe_notifier_worker ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



128
129
130
 
# File 'lib/datadog/di/probe_manager.rb', line 128

def probe_notifier_worker
  @probe_notifier_worker
end

#probe_repository ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



43
44
45
 
# File 'lib/datadog/di/probe_manager.rb', line 43

def probe_repository
  @probe_repository
end

#settings ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



125
126
127
 
# File 'lib/datadog/di/probe_manager.rb', line 125

def settings
  @settings
end

#telemetry ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



42
43
44
 
# File 'lib/datadog/di/probe_manager.rb', line 42

def telemetry
  @telemetry
end

Instance Method Details

#add_probe(probe) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Requests to install the specified probe.

If the target of the probe does not exist, assume the relevant code is not loaded yet (rather than that it will never be loaded), and store the probe in a pending probe list. When classes are defined, or files loaded, the probe will be checked against the newly defined classes/loaded files, and will be installed if it matches.

On successful installation, sends INSTALLED status to the backend. On failure, sends ERROR status to the backend before re-raising.

Parameters:

  • probe (Probe) —

    the probe to install

Returns:

  • (Boolean) —

    true if installed, false if pending

Raises:



147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/datadog/di/probe_manager.rb', line 147

def add_probe(probe)
  probe_repository.synchronize do
    if probe_repository.find_installed(probe.id)
      # Either this probe was already installed, or another probe was
      # installed with the same id (previous version perhaps?).
      # Since our state tracking is keyed by probe id, we cannot
      # install this probe since we won't have a way of removing the
      # instrumentation for the probe with the same id which is already
      # installed.
      #
      # The exception raised here will be caught below and logged and
      # reported to telemetry.
      raise Error::AlreadyInstrumented, "Probe with id #{probe.id} is already in installed probes"
    end

    # Probe failed to install previously, do not try to install it again.
    if msg = probe_repository.find_failed(probe.id)
      # TODO test this path
      raise Error::ProbePreviouslyFailed, msg
    end

    begin
      instrumenter.hook(probe, self)

      probe_repository.add_installed(probe)
      payload = probe_notification_builder.build_installed(probe)
      probe_notifier_worker.add_status(payload, probe: probe)
      # The probe would only be in the pending probes list if it was
      # previously attempted to be installed and the target was not loaded.
      # Always remove from pending list here because it makes the
      # API smaller and shouldn't cause any actual problems.
      probe_repository.remove_pending(probe.id)
      logger.trace { "di: installed #{probe.type} probe at #{probe.location} (#{probe.id})" }
      true
    rescue Error::DITargetNotDefined
      probe_repository.add_pending(probe)
      logger.trace { "di: could not install #{probe.type} probe at #{probe.location} (#{probe.id}) because its target is not defined, adding it to pending list" }
      false
    end
  end
rescue Exception => exc # standard:disable Lint/RescueException
  Datadog::DI.reraise_if_fatal(exc)
  # In "propagate all exceptions" mode we will try to instrument again.
  raise if settings.dynamic_instrumentation.internal.propagate_all_exceptions

  logger.debug { "di: error processing probe configuration: #{exc.class}: #{exc.message}" }
  telemetry&.report(exc, description: "Error processing probe configuration")

  payload = probe_notification_builder.build_errored(probe, exc)
  probe_notifier_worker.add_status(payload, probe: probe)

  probe_repository.add_failed(probe.id, "#{exc.class}: #{exc.message}")

  raise
end

#clear_hooks ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Removes all installed probe instrumentation and clears the probe repository.

Iterates through all installed probes, unhooks their instrumentation, and clears all probe collections (installed, pending, failed). Called during component shutdown to clean up resources.



119
120
121
122
123
 
# File 'lib/datadog/di/probe_manager.rb', line 119

def clear_hooks
  probe_repository.clear_all do |probe|
    instrumenter.unhook(probe)
  end
end

#close ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Shuts down the probe manager and releases all resources.

Disables the class definition trace point and removes all installed probe instrumentation. Called during component teardown. Unlike #stop, this is not reversible.



107
108
109
110
 
# File 'lib/datadog/di/probe_manager.rb', line 107

def close
  definition_trace_point.disable
  clear_hooks
end

#install_pending_line_probes(path) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Installs pending line probes, if any, for the file of the specified absolute path.

This method is meant to be called from the script_compiled trace point, which is invoked for each required or loaded file (and also for eval'd code, but those invocations are filtered out).



274
275
276
277
278
279
280
281
282
283
284
285
286
287
 
# File 'lib/datadog/di/probe_manager.rb', line 274

def install_pending_line_probes(path)
  if path.nil?
    raise ArgumentError, "path must not be nil"
  end
  probe_repository.synchronize do
    probe_repository.pending_probes.values.each do |probe|
      if probe.line?
        if probe.file_matches?(path)
          add_probe(probe)
        end
      end
    end
  end
end

#probe_condition_evaluation_failed_callback(context, expr, exc) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Callback invoked when a probe's condition expression fails to evaluate.

This can happen when the expression references undefined variables, has type mismatches, or encounters runtime errors during evaluation. Rate-limited to 1 notification per second per probe to avoid flooding the backend when conditions fail repeatedly.

Parameters:

  • context (Context) —

    The execution context containing probe and captured data

  • expr (String) —

    The condition expression that failed

  • exc (Exception) —

    The exception raised during condition evaluation



319
320
321
322
323
324
325
 
# File 'lib/datadog/di/probe_manager.rb', line 319

def probe_condition_evaluation_failed_callback(context, expr, exc)
  probe = context.probe
  if probe.condition_evaluation_failed_rate_limiter&.allow?
    payload = probe_notification_builder.build_condition_evaluation_failed(context, expr, exc)
    probe_notifier_worker.add_snapshot(payload)
  end
end

#probe_disabled_callback(probe, duration) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Callback invoked when a probe is disabled, for example due to exceeding the CPU consumption limit in DI processing.

Sends ERROR status notification to the backend.

Parameters:

  • probe (Probe) —

    The probe that was disabled

  • duration (Numeric) —

    The CPU time consumed by probe processing, in seconds



334
335
336
337
 
# File 'lib/datadog/di/probe_manager.rb', line 334

def probe_disabled_callback(probe, duration)
  payload = probe_notification_builder.build_disabled(probe, duration)
  probe_notifier_worker.add_status(payload, probe: probe)
end

#probe_executed_callback(context) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Entry point invoked from the instrumentation when the specfied probe is invoked (that is, either its target method is invoked, or execution reached its target file/line).

This method is responsible for queueing probe status to be sent to the backend (once per the probe's lifetime) and a snapshot corresponding to the current invocation.



296
297
298
299
300
301
302
303
304
305
306
307
 
# File 'lib/datadog/di/probe_manager.rb', line 296

def probe_executed_callback(context)
  probe = context.probe
  logger.trace { "di: executed #{probe.type} probe at #{probe.location} (#{probe.id})" }
  unless probe.emitting_notified?
    payload = probe_notification_builder.build_emitting(probe)
    probe_notifier_worker.add_status(payload, probe: probe)
    probe.emitting_notified = true
  end

  payload = probe_notification_builder.build_executed(context)
  probe_notifier_worker.add_snapshot(payload)
end

#remove_probe(probe_id) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Removes probe with specified id. The probe could be pending or installed. Does nothing if there is no probe with the specified id.



205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/datadog/di/probe_manager.rb', line 205

def remove_probe(probe_id)
  probe_repository.remove_pending(probe_id)

  # Do not delete the probe from the registry here in case
  # deinstrumentation fails - though I don't know why deinstrumentation
  # would fail and how we could recover if it does.
  # I plan on tracking the number of outstanding (instrumented) probes
  # in the future, and if deinstrumentation fails I would want to
  # keep that probe as "installed" for the count, so that we can
  # investigate the situation.
  if probe = probe_repository.find_installed(probe_id)
    begin
      instrumenter.unhook(probe)
      probe_repository.remove_installed(probe_id)
    rescue Exception => exc # standard:disable Lint/RescueException
      Datadog::DI.reraise_if_fatal(exc)
      raise if settings.dynamic_instrumentation.internal.propagate_all_exceptions
      # Silence all exceptions?
      # TODO should we propagate here and rescue upstream?
      logger.debug { "di: error removing #{probe.type} probe at #{probe.location} (#{probe.id}): #{exc.class}: #{exc.message}" }
      telemetry&.report(exc, description: "Error removing probe")
    end
  end
end

#reopen ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Re-enables the probe manager after a #stop.

Re-enables the class definition trace point so that pending method probes can be installed when their target classes are defined, and re-hooks all previously-installed probes from the repository. If a probe's target is no longer present (e.g. code was unloaded between stop and reopen), it is demoted to pending so the trace point or code tracker can re-install it if the target reappears.



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/datadog/di/probe_manager.rb', line 81

def reopen
  definition_trace_point.enable
  probe_repository.synchronize do
    # values.each so concurrent mutation of @installed_probes (via
    # remove_installed below) does not iterate over a stale key set.
    probe_repository.installed_probes.values.each do |probe|
      instrumenter.hook(probe, self)
    rescue Error::DITargetNotDefined
      probe_repository.remove_installed(probe.id)
      probe_repository.add_pending(probe)
      logger.trace { "di: target for #{probe.type} probe (#{probe.id}) no longer defined on reopen, demoting to pending" }
    rescue => exc
      raise if settings.dynamic_instrumentation.internal.propagate_all_exceptions
      logger.debug { "di: error re-hooking #{probe.type} probe at #{probe.location} (#{probe.id}) on reopen: #{exc.class}: #{exc.message}" }
      telemetry&.report(exc, description: "Error re-hooking probe on reopen")
    end
  end
end

#stop ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Stops the probe manager without permanently releasing resources.

Disables the class definition trace point and unhooks all installed probe instrumentation, but preserves the probe repository so that #reopen can re-install probes. Can be reversed by calling #reopen.

Why preserve the repository: RC dispatch only delivers LIVE_DEBUGGING changes when the content hash changes. If a customer toggles DI off then back on in the UI with the same probe content, the receiver is not re-invoked, so probes wiped here would never be redelivered. Preserving the repo lets #reopen replay the probes locally.



58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/datadog/di/probe_manager.rb', line 58

def stop
  definition_trace_point.disable
  probe_repository.synchronize do
    probe_repository.installed_probes.each_value do |probe|
      instrumenter.unhook(probe)
    rescue => exc
      raise if settings.dynamic_instrumentation.internal.propagate_all_exceptions
      logger.debug { "di: error unhooking #{probe.type} probe at #{probe.location} (#{probe.id}) on stop: #{exc.class}: #{exc.message}" }
      telemetry&.report(exc, description: "Error unhooking probe on stop")
    end
  end
end