Class: DogTrainer::API

Inherits:
Object
  • Object
show all
Includes:
Logging
Defined in:
lib/dogtrainer/api.rb

Overview

Helper methods to upsert/ensure existence and configuration of DataDog Monitors, TimeBoards and ScreenBoards.

Instance Method Summary collapse

Methods included from Logging

debug_formatter, default_formatter, default_outputter, #logger, #logger_name

Constructor Details

#initialize(api_key, app_key, notify_to, repo_path = nil) ⇒ API

Initialize class; set instance configuration.

Parameters:

  • api_key (String) —

    DataDog API Key

  • app_key (String) —

    DataDog Application Key

  • notify_to (String) —

    DataDog notification recpipent string for monitors. This is generally one or more @-prefixed DataDog users or notification recipients. It can be set to nil if you are only managing screenboards and timeboards. For further information, see: docs.datadoghq.com/monitoring/#notifications

  • repo_path (String) (defaults to: nil) —

    Git or HTTP URL to the repository containing code that calls this class. Will be added to notification messages so that humans know where to make changes to monitors. If nil, the return value of #get_repo_path



25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/dogtrainer/api.rb', line 25

def initialize(api_key, app_key, notify_to, repo_path = nil)
  logger.debug 'initializing DataDog API client'
  @dog = Dogapi::Client.new(api_key, app_key)
  @monitors = nil
  @timeboards = nil
  @screenboards = nil
  @notify_to = notify_to
  if repo_path.nil?
    @repo_path = get_repo_path
    logger.debug "using repo_path: #{@repo_path}"
  else
    @repo_path = repo_path
  end
end

Instance Method Details

#check_dog_result(r, accepted_codes = ['200']) ⇒ Object

Check the result of a Dogapi::Client call.

Dogapi::Client returns responses as arrays, with the first element being the HTTP response code and the second element being the actual response.

Check the specified

Parameters:

  • r (Array) —

    the Dogapi result/response

  • accepted_codes (Array) (defaults to: ['200']) —

    Array of acceptable (success) HTTP codes

Raises:



50
51
52
 
# File 'lib/dogtrainer/api.rb', line 50

def check_dog_result(r, accepted_codes = ['200'])
  raise DogApiException, r unless accepted_codes.include?(r[0])
end

#create_monitor(_mon_name, mon_params) ⇒ Object

Create a monitor that doesn’t already exist; return its id

Parameters:

  • _mon_name (String) —

    mane of the monitor to create

  • mon_params (Hash) —

    params to pass to the DataDog API call. Must include “type” and “query” keys.



361
362
363
364
365
366
367
368
369
 
# File 'lib/dogtrainer/api.rb', line 361

def create_monitor(_mon_name, mon_params)
  res = @dog.monitor(mon_params['type'], mon_params['query'], mon_params)
  if res[0] == '200'
    logger.info "\tMonitor #{res[1]['id']} created successfully"
    return res[1]['id']
  else
    logger.error "\tError creating monitor: #{res}"
  end
end

#generate_messages(metric_desc, comparison, mon_type) ⇒ Object

Given the name of a metric we’re monitoring and the comparison method, generate alert messages for the monitor.

This method is intended for internal use by the class, but can be overridden if the implementation is not desired.

Parameters:

  • metric_desc (String) —

    description/name of the metric being monitored.

  • comparison (String) —

    comparison operator or description for metric vs threshold; i.e. “>=”, “<=”, “=”, “<”, etc.

  • mon_type (Hash) —

    a customizable set of options

Options Hash (mon_type):

  • type (String) —

    of monitor as defined in DataDog API docs.



117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/dogtrainer/api.rb', line 117

def generate_messages(metric_desc, comparison, mon_type)
  if mon_type == 'service check'
    message = [
      "{{#is_alert}}'#{metric_desc}' is FAILING: {{check_message}}",
      "{{/is_alert}}\n",
      "{{#is_warning}}'#{metric_desc}' is WARNING: {{check_message}}",
      "{{/is_warning}}\n",
      "{{#is_recovery}}'#{metric_desc}' recovered: {{check_message}}",
      "{{/is_recovery}}\n",
      "{{#is_no_data}}'#{metric_desc}' is not reporting data",
      "{{/is_no_data}}\n",
      # repo path and notify to
      '(monitor and threshold configuration for this alert is managed by ',
      "#{@repo_path}) #{@notify_to}"
    ].join('')
    escalation = "'#{metric_desc}' is still in error state: " \
      '{{check_message}}'
    return [message, escalation]
  end
  message = [
    "{{#is_alert}}'#{metric_desc}' should be #{comparison} {{threshold}}, ",
    "but is {{value}}.{{/is_alert}}\n",
    "{{#is_recovery}}'#{metric_desc}' recovered  (current value {{value}} ",
    "is #{comparison} threshold of {{threshold}}).{{/is_recovery}}\n",
    '(monitor and threshold configuration for this alert is managed by ',
    "#{@repo_path}) #{@notify_to}"
  ].join('')
  escalation = "'#{metric_desc}' is still in error state (current value " \
    "{{value}} is #{comparison} threshold of {{threshold}})"
  [message, escalation]
end

#get_existing_monitor_by_name(mon_name) ⇒ Object

Get all monitors from DataDog; return the one named “mon_name“ or nil

This caches all monitors from DataDog in an instance variable.

Parameters:

  • mon_name (String) —

    name of the monitor to return



376
377
378
379
380
381
 
# File 'lib/dogtrainer/api.rb', line 376

def get_existing_monitor_by_name(mon_name)
  get_monitors.each do |mon|
    return mon if mon['name'] == mon_name
  end
  nil
end

#get_existing_screenboard_by_name(dash_name) ⇒ Object

get all screenboards from DataDog; return the one named “dash_name“ or nil returns the screenboard definition hash from the DataDog API



688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
 
# File 'lib/dogtrainer/api.rb', line 688

def get_existing_screenboard_by_name(dash_name)
  if @screenboards.nil?
    @screenboards = @dog.get_all_screenboards
    puts "Found #{@screenboards[1]['screenboards'].length} existing " \
      'screenboards in DataDog'
    if @screenboards[1]['screenboards'].empty?
      puts 'ERROR: Docker API call returned no existing screenboards. ' \
        'Something is wrong.'
      exit 1
    end
  end
  @screenboards[1]['screenboards'].each do |dash|
    return @dog.get_screenboard(dash['id'])[1] if dash['title'] == dash_name
  end
  nil
end

#get_existing_timeboard_by_name(dash_name) ⇒ Object

get all timeboards from DataDog; return the one named “dash_name“ or nil returns the timeboard definition hash from the DataDog API



669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
 
# File 'lib/dogtrainer/api.rb', line 669

def get_existing_timeboard_by_name(dash_name)
  if @timeboards.nil?
    @timeboards = @dog.get_dashboards
    puts "Found #{@timeboards[1]['dashes'].length} existing timeboards " \
      'in DataDog'
    if @timeboards[1]['dashes'].empty?
      puts 'ERROR: Docker API call returned no existing timeboards. ' \
        'Something is wrong.'
      exit 1
    end
  end
  @timeboards[1]['dashes'].each do |dash|
    return @dog.get_dashboard(dash['id'])[1] if dash['title'] == dash_name
  end
  nil
end

#get_git_url_for_directory(dir_path) ⇒ Object

Given the path to a directory on disk that may be a git repository, return the URL to its first remote, or nil otherwise.

Parameters:

  • dir_path (String) —

    Path to possible git repository



84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
 
# File 'lib/dogtrainer/api.rb', line 84

def get_git_url_for_directory(dir_path)
  logger.debug "trying to find git remote for: #{dir_path}"
  conf = nil
  Dir.chdir(dir_path) do
    begin
      conf = `git config --local -l`
    rescue
      conf = nil
    end
  end
  return nil if conf.nil?
  conf.split("\n").each do |line|
    return Regexp.last_match(1) if line =~ /^remote\.[^\.]+\.url=(.+)/
  end
  nil
end

#get_monitors ⇒ Object

Get all monitors from DataDog, caching them in an instance variable.



384
385
386
387
388
389
390
391
392
393
394
 
# File 'lib/dogtrainer/api.rb', line 384

def get_monitors
  if @monitors.nil?
    @monitors = @dog.get_all_monitors(group_states: 'all')
    logger.info "Found #{@monitors[1].length} existing monitors in DataDog"
    if @monitors[1].empty?
      raise 'ERROR: DataDog API call returned no existing monitors. ' \
        'Something is wrong.'
    end
  end
  @monitors[1]
end

#get_repo_path ⇒ Object

Return a human-usable string identifying where to make changes to the resources created by this class. Returns the first of:

  1. “GIT_URL“ environment variable, if set and not empty

  2. “CIRCLE_REPOSITORY_URL“ environment variable, if set and not empty

  3. If the code calling this class is part of a git repository on disk and “git“ is present on the system and in PATH, the URL of the first remote for the repository.

If none of these are found, an error will be raised.



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/dogtrainer/api.rb', line 64

def get_repo_path
  %w[GIT_URL CIRCLE_REPOSITORY_URL].each do |vname|
    return ENV[vname] if ENV.has_key?(vname) && !ENV[vname].empty?
  end
  # try to find git repository
  # get the path to the calling code;
  #   caller[0] is #initialize, caller[1] is what instantiated the class
  path, = caller[1].partition(':')
  repo_path = get_git_url_for_directory(File.dirname(path))
  if repo_path.nil?
    raise 'Unable to determine source code path; please ' \
    'specify repo_path option to DogTrainer::API'
  end
  repo_path
end

#graphdef(title, queries, markers = {}) ⇒ Object

Create a graph definition (graphdef) to use with Boards APIs. For further information, see: docs.datadoghq.com/graphingjson/

Parameters:

  • title (String) —

    title of the graph

  • queries (Array or String) —

    a single string graph query, or an Array of graph query strings.

  • markers (Hash) (defaults to: {}) —

    a hash of markers to set on the graph, in name => value format.



544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
 
# File 'lib/dogtrainer/api.rb', line 544

def graphdef(title, queries, markers = {})
  queries = [queries] unless queries.is_a?(Array)
  d = {
    'definition' => {
      'viz' => 'timeseries',
      'requests' => []
    },
    'title' => title
  }
  queries.each do |q|
    d['definition']['requests'] << {
      'q' => q,
      'conditional_formats' => [],
      'type' => 'line'
    }
  end
  unless markers.empty?
    d['definition']['markers'] = []
    markers.each do |name, val|
      d['definition']['markers'] << {
        'type' => 'error dashed',
        'val' => val.to_s,
        'value' => "y = #{val}",
        'label' => "#{name}==#{val}"
      }
    end
  end
  d
end

#mute_monitor_by_id(mon_id, options = { end_timestamp: nil }) ⇒ Object

Mute the monitor identified by the specified unique ID, with an optional duration.

Examples:

mute monitor 12345 indefinitely

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_id(12345)

mute monitor 12345 until 2016-09-17 01:39:52-00:00

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_id(12345, end_timestamp: 1474076393)

Parameters:

  • mon_id (Integer) —

    ID of the monitor to mute

  • options (Hash) (defaults to: { end_timestamp: nil })

Options Hash (options):

  • :end_timestamp (Integer) —

    optional timestamp for when the mute should end; Integer POSIX timestamp.

Raises:



412
413
414
415
416
417
418
419
420
421
 
# File 'lib/dogtrainer/api.rb', line 412

def mute_monitor_by_id(mon_id, options = { end_timestamp: nil })
  if options.fetch(:end_timestamp, nil).nil?
    logger.info "Muting monitor by ID #{mon_id}"
    check_dog_result(@dog.mute_monitor(mon_id))
  else
    end_ts = options[:end_timestamp]
    logger.info "Muting monitor by ID #{mon_id} until #{end_ts}"
    check_dog_result(@dog.mute_monitor(mon_id, end: end_ts))
  end
end

#mute_monitor_by_name(mon_name, options = { end_timestamp: nil }) ⇒ Object

Mute the monitor identified by the specified name, with an optional duration.

Examples:

mute monitor named ‘My Monitor’ indefinitely

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_name('My Monitor')

mute monitor named ‘My Monitor’ until 2016-09-17 01:39:52-00:00

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_name('My Monitor', end_timestamp: 1474076393)

Parameters:

  • mon_name (String) —

    name of the monitor to mute

  • options (Hash) (defaults to: { end_timestamp: nil })

Options Hash (options):

  • :end_timestamp (Integer) —

    optional timestamp for when the mute should end; Integer POSIX timestamp.

Raises:

  • (RuntimeError) —

    raised if the specified monitor name can’t be found

  • (DogApiException) —

    if the Datadog API returns an error



440
441
442
443
444
445
446
447
448
449
450
451
452
 
# File 'lib/dogtrainer/api.rb', line 440

def mute_monitor_by_name(mon_name, options = { end_timestamp: nil })
  mon = get_existing_monitor_by_name(mon_name)
  raise "ERROR: Could not find monitor with name #{mon_name}" if mon.nil?
  if options.fetch(:end_timestamp, nil).nil?
    logger.info "Muting monitor by name #{mon_name} (#{mon['id']})"
    check_dog_result(@dog.mute_monitor(mon['id']))
  else
    end_ts = options[:end_timestamp]
    logger.info "Muting monitor by name #{mon_name} (#{mon['id']}) " \
      "until #{end_ts}"
    check_dog_result(@dog.mute_monitor(mon['id'], end: end_ts))
  end
end

#mute_monitors_by_regex(mon_name_regex, options = { end_timestamp: nil }) ⇒ Object

Mute all monitors with names matching the specified regex, with an optional duration.

Examples:

mute monitors with names matching /myapp/ indefinitely

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_regex(/myapp/)

mute monitors with names containing ‘foo’ indefinitely

dog = DogTrainer::API.new(api_key, app_key, notify_to)
dog.mute_monitor_by_regex('foo')

mute monitors with names matching /myapp/ until 2016-09-17

01:39:52-00:00
 dog = DogTrainer::API.new(api_key, app_key, notify_to)
 dog.mute_monitor_by_regex(/myapp/, end_timestamp: 1474076393)

Parameters:

  • mon_name_regex (String) —

    or [Regexp] regex to match monitor names against

  • options (Hash) (defaults to: { end_timestamp: nil })

Options Hash (options):

  • :end_timestamp (Integer) —

    optional timestamp for when the mute should end; Integer POSIX timestamp.



475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
 
# File 'lib/dogtrainer/api.rb', line 475

def mute_monitors_by_regex(mon_name_regex, options = { end_timestamp: nil })
  if mon_name_regex.class != Regexp
    mon_name_regex = Regexp.new(mon_name_regex)
  end
  if options.fetch(:end_timestamp, nil).nil?
    logger.info "Muting monitors by regex #{mon_name_regex.source}"
    end_ts = nil
  else
    logger.info "Muting monitors by regex #{mon_name_regex.source} " \
      "until #{end_ts}"
    end_ts = options[:end_timestamp]
  end
  logger.debug "Searching for monitors matching: #{mon_name_regex.source}"
  get_monitors.each do |mon|
    if mon['name'] =~ mon_name_regex
      logger.info "Muting monitor '#{mon['name']}' (#{mon['id']})"
      mute_monitor_by_id(mon['id'], end_timestamp: end_ts)
    end
  end
end

#params_for_monitor(name, message, query, threshold, options = { escalation_message: nil, alert_no_data: true, mon_type: 'metric alert', renotify_interval: 60, no_data_timeframe: 20, evaluation_delay: nil }) ⇒ Object

Return a hash of parameters for a monitor with the specified configuration. For further information, see: docs.datadoghq.com/api/#monitors

Parameters:

  • name (String) —

    name for the monitor; must be unique per DataDog account

  • message (String) —

    alert/notification message for the monitor

  • query (String) —

    query for the monitor to evaluate

  • threshold (Float or Hash) —

    evaluation threshold for the monitor; if a Float is passed, it will be provided as the “critical“ threshold; otherise, a Hash in the form taken by the DataDog API should be provided (“critical“, “warning“ and/or “ok“ keys, Float values)

  • options (Hash) (defaults to: { escalation_message: nil, alert_no_data: true, mon_type: 'metric alert', renotify_interval: 60, no_data_timeframe: 20, evaluation_delay: nil })

Options Hash (options):

  • :escalation_message (String) —

    optional escalation message for escalation notifications. Defaults to nil.

  • :alert_no_data (Boolean) —

    whether or not to alert on lack of data. Defaults to true.

  • :mon_type (String) —

    type of monitor as defined in DataDog API docs. Defaults to ‘metric alert’.

  • :renotify_interval (Integer) —

    the number of minutes after the last notification before a monitor will re-notify on the current status. It will re-notify only if not resolved. Default: 60. Set to nil to disable re-notification.

  • :no_data_timeframe (Integer) —

    the number of minutes before a monitor will notify when data stops reporting. Must be at least 2x the monitor timeframe for metric alerts or 2 minutes for service checks. Defaults to 20 minutes; API default is 2x the monitor timeframe.

  • :evaluation_delay (Integer) — default: metric monitors only —

    Time (in seconds) to delay evaluation, as a non-negative integer. For example, if the value is set to 300 (5min), the timeframe is set to last_5m and the time is 7:00, the monitor will evaluate data from 6:50 to 6:55. This is useful for AWS CloudWatch and other backfilled metrics to ensure the monitor will always have data during evaluation.



182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/dogtrainer/api.rb', line 182

def params_for_monitor(
  name,
  message,
  query,
  threshold,
  options = {
    escalation_message: nil,
    alert_no_data: true,
    mon_type: 'metric alert',
    renotify_interval: 60,
    no_data_timeframe: 20,
    evaluation_delay: nil
  }
)
  options[:alert_no_data] = true unless options.key?(:alert_no_data)
  options[:mon_type] = 'metric alert' unless options.key?(:mon_type)
  options[:renotify_interval] = 60 unless options.key?(:renotify_interval)
  options[:no_data_timeframe] = 20 unless options.key?(:no_data_timeframe)
  options[:evaluation_delay] = nil unless options.key?(:evaluation_delay)

  # handle threshold hash
  thresh = if threshold.is_a?(Hash)
             threshold
           else
             { 'critical' => threshold }
           end

  monitor_data = {
    'name' => name,
    'type' => options[:mon_type],
    'query' => query,
    'message' => message,
    'tags' => [],
    'options' => {
      'notify_audit' => false,
      'locked' => false,
      'timeout_h' => 0,
      'silenced' => {},
      'thresholds' => thresh,
      'require_full_window' => false,
      'notify_no_data' => options[:alert_no_data],
      'renotify_interval' => options[:renotify_interval],
      'no_data_timeframe' => options[:no_data_timeframe]
    }
  }
  unless options[:escalation_message].nil?
    monitor_data['options']['escalation_message'] = \
      options[:escalation_message]
  end
  unless options[:evaluation_delay].nil?
    monitor_data['options']['evaluation_delay'] = options[:evaluation_delay]
  end
  monitor_data
end

#unmute_monitor_by_id(mon_id) ⇒ Object

Unute the monitor identified by the specified unique ID.

Parameters:

  • mon_id (Integer) —

    ID of the monitor to mute

Raises:



500
501
502
503
 
# File 'lib/dogtrainer/api.rb', line 500

def unmute_monitor_by_id(mon_id)
  logger.info "Unmuting monitor by ID #{mon_id}"
  check_dog_result(@dog.unmute_monitor(mon_id, all_scopes: true))
end

#unmute_monitor_by_name(mon_name) ⇒ Object

Unmute the monitor identified by the specified name.

Parameters:

  • mon_name (String) —

    name of the monitor to mute

Raises:

  • (RuntimeError) —

    raised if the specified monitor name can’t be found



509
510
511
512
513
514
 
# File 'lib/dogtrainer/api.rb', line 509

def unmute_monitor_by_name(mon_name)
  mon = get_existing_monitor_by_name(mon_name)
  logger.info "Unmuting monitor by name #{mon_name}"
  raise "ERROR: Could not find monitor with name #{mon_name}" if mon.nil?
  unmute_monitor_by_id(mon['id'])
end

#unmute_monitors_by_regex(mon_name_regex) ⇒ Object

Unmute all monitors with names matching the specified regex.

Parameters:

  • mon_name_regex (String) —

    regex to match monitor names against



519
520
521
522
523
524
525
526
527
528
529
530
 
# File 'lib/dogtrainer/api.rb', line 519

def unmute_monitors_by_regex(mon_name_regex)
  if mon_name_regex.class != Regexp
    mon_name_regex = Regexp.new(mon_name_regex)
  end
  logger.info "Unmuting monitors by regex #{mon_name_regex.source}"
  get_monitors.each do |mon|
    if mon['name'] =~ mon_name_regex
      logger.info "Unmuting monitor '#{mon['name']}' (#{mon['id']})"
      unmute_monitor_by_id(mon['id'])
    end
  end
end

#upsert_monitor(mon_name, query, threshold, comparator, options = { alert_no_data: true, mon_type: 'metric alert', renotify_interval: 60, no_data_timeframe: 20, evaluation_delay: nil, message: nil }) ⇒ Object

Create or update a monitor in DataDog with the given name and data/params. This method handles either creating the monitor if one with the same name doesn’t already exist in the specified DataDog account, or else updating an existing monitor with the same name if one exists but the parameters differ.

For further information on parameters and options, see: docs.datadoghq.com/api/#monitors

This method calls #generate_messages to build the notification messages and #params_for_monitor to generate the parameters.

Parameters:

  • mon_name (String) —

    name for the monitor; must be unique per DataDog account

  • query (String) —

    query for the monitor to evaluate

  • threshold (Float or Hash) —

    evaluation threshold for the monitor; if a Float is passed, it will be provided as the “critical“ threshold; otherise, a Hash in the form taken by the DataDog API should be provided (“critical“, “warning“ and/or “ok“ keys, Float values)

  • comparator (String) —

    comparison operator for metric vs threshold, describing the inverse of the query. I.e. if the query is checking for “< 100”, then the comparator would be “>=”.

  • options (Hash) (defaults to: { alert_no_data: true, mon_type: 'metric alert', renotify_interval: 60, no_data_timeframe: 20, evaluation_delay: nil, message: nil })

Options Hash (options):

  • :alert_no_data (Boolean) —

    whether or not to alert on lack of data. Defaults to true.

  • :mon_type (String) —

    type of monitor as defined in DataDog API docs. Defaults to ‘metric alert’.

  • :renotify_interval (Integer) —

    the number of minutes after the last notification before a monitor will re-notify on the current status. It will re-notify only if not resolved. Default: 60. Set to nil to disable re-notification.

  • :no_data_timeframe (Integer) —

    the number of minutes before a monitor will notify when data stops reporting. Must be at least 2x the monitor timeframe for metric alerts or 2 minutes for service checks. Defaults to 20 minutes; API default is 2x the monitor timeframe.

  • :evaluation_delay (Integer) — default: metric monitors only —

    Time (in seconds) to delay evaluation, as a non-negative integer. For example, if the value is set to 300 (5min), the timeframe is set to last_5m and the time is 7:00, the monitor will evaluate data from 6:50 to 6:55. This is useful for AWS CloudWatch and other backfilled metrics to ensure the monitor will always have data during evaluation.

  • :message (String) —

    alert/notification message for the monitor; if omitted, will be generated by #generate_messages

  • :escalation_message (String) —

    optional escalation message for escalation notifications. If omitted, will be generated by #generate_messages; explicitly set to nil to not add an escalation message to the monitor.



284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
 
# File 'lib/dogtrainer/api.rb', line 284

def upsert_monitor(
  mon_name,
  query,
  threshold,
  comparator,
  options = {
    alert_no_data: true,
    mon_type: 'metric alert',
    renotify_interval: 60,
    no_data_timeframe: 20,
    evaluation_delay: nil,
    message: nil
  }
)
  options[:alert_no_data] = true unless options.key?(:alert_no_data)
  options[:mon_type] = 'metric alert' unless options.key?(:mon_type)
  options[:renotify_interval] = 60 unless options.key?(:renotify_interval)
  options[:no_data_timeframe] = 20 unless options.key?(:no_data_timeframe)
  options[:evaluation_delay] = nil unless options.key?(:evaluation_delay)

  msg, esc = generate_messages(mon_name, comparator, options[:mon_type])
  message = if options[:message].nil?
              msg
            else
              options[:message]
            end
  escalation = if options.key?(:escalation_message)
                 options[:escalation_message]
               else
                 esc
               end

  rno = options[:renotify_interval]
  mon_params = params_for_monitor(
    mon_name, message, query, threshold,
    escalation_message: escalation,
    alert_no_data: options[:alert_no_data],
    mon_type: options[:mon_type],
    renotify_interval: rno,
    no_data_timeframe: options[:no_data_timeframe],
    evaluation_delay: options[:evaluation_delay]
  )
  logger.info "Upserting monitor: #{mon_name}"
  monitor = get_existing_monitor_by_name(mon_name)
  return create_monitor(mon_name, mon_params) if monitor.nil?
  logger.debug "\tfound existing monitor id=#{monitor['id']}"
  do_update = false
  mon_params.each do |k, _v|
    unless monitor.include?(k)
      logger.debug "\tneeds update based on missing key: #{k}"
      do_update = true
      break
    end
    next unless monitor[k] != mon_params[k]
    logger.debug "\tneeds update based on difference in key #{k}; " \
      "current='#{monitor[k]}' desired='#{mon_params[k]}'"
    do_update = true
    break
  end
  unless do_update
    logger.debug "\tmonitor is correct in DataDog."
    return monitor['id']
  end
  res = @dog.update_monitor(monitor['id'], mon_params['query'], mon_params)
  if res[0] == '200'
    logger.info "\tMonitor #{monitor['id']} updated successfully"
    return monitor['id']
  else
    logger.error "\tError updating monitor #{monitor['id']}: #{res}"
  end
end

#upsert_screenboard(dash_name, widgets) ⇒ Object

Create or update a screenboard in DataDog with the given name and data/params. For further information, see: docs.datadoghq.com/api/screenboards/ and docs.datadoghq.com/api/?lang=ruby#screenboards

Parameters:

  • dash_name (String) —

    Account-unique dashboard name

  • widgets (Array) —

    Array of Hash widget definitions to pass to the DataDog API. For further information, see: docs.datadoghq.com/api/screenboards/

Raises:



628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
 
# File 'lib/dogtrainer/api.rb', line 628

def upsert_screenboard(dash_name, widgets)
  logger.info "Upserting screenboard: #{dash_name}"
  desc = "created by DogTrainer RubyGem via #{@repo_path}"
  dash = get_existing_screenboard_by_name(dash_name)
  if dash.nil?
    d = @dog.create_screenboard(board_title: dash_name,
                                description: desc,
                                widgets: widgets)
    check_dog_result(d)
    logger.info "Created screenboard #{d[1]['id']}"
    return
  end
  logger.debug "\tfound existing screenboard id=#{dash['id']}"
  needs_update = false
  if dash['description'] != desc
    logger.debug "\tneeds update of description"
    needs_update = true
  end
  if dash['board_title'] != dash_name
    logger.debug "\tneeds update of title"
    needs_update = true
  end
  if dash['widgets'] != widgets
    logger.debug "\tneeds update of widgets"
    needs_update = true
  end

  if needs_update
    logger.info "\tUpdating screenboard #{dash['id']}"
    d = @dog.update_screenboard(dash['id'], board_title: dash_name,
                                            description: desc,
                                            widgets: widgets)
    check_dog_result(d)
    logger.info "\tScreenboard updated."
  else
    logger.info "\tScreenboard is up-to-date"
  end
end

#upsert_timeboard(dash_name, graphs) ⇒ Object

Create or update a timeboard in DataDog with the given name and data/params. For further information, see: docs.datadoghq.com/api/#timeboards

Parameters:

  • dash_name (String) —

    Account-unique dashboard name

  • graphs (Array) —

    Array of graphdefs to add to dashboard

Raises:



581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
 
# File 'lib/dogtrainer/api.rb', line 581

def upsert_timeboard(dash_name, graphs)
  logger.info "Upserting timeboard: #{dash_name}"
  desc = "created by DogTrainer RubyGem via #{@repo_path}"
  dash = get_existing_timeboard_by_name(dash_name)
  if dash.nil?
    d = @dog.create_dashboard(dash_name, desc, graphs)
    check_dog_result(d)
    logger.info "Created timeboard #{d[1]['dash']['id']}"
    return
  end
  logger.debug "\tfound existing timeboard id=#{dash['dash']['id']}"
  needs_update = false
  if dash['dash']['description'] != desc
    logger.debug "\tneeds update of description"
    needs_update = true
  end
  if dash['dash']['title'] != dash_name
    logger.debug "\tneeds update of title"
    needs_update = true
  end
  if dash['dash']['graphs'] != graphs
    logger.debug "\tneeds update of graphs"
    needs_update = true
  end

  if needs_update
    logger.info "\tUpdating timeboard #{dash['dash']['id']}"
    d = @dog.update_dashboard(
      dash['dash']['id'], dash_name, desc, graphs
    )
    check_dog_result(d)
    logger.info "\tTimeboard updated."
  else
    logger.info "\tTimeboard is up-to-date"
  end
end