Class: ConfigSkeleton

Inherits:
Object
  • Object
show all
Includes:
ServiceSkeleton
Defined in:
lib/config_skeleton.rb

Overview

Framework for creating config generation systems.

There are many systems which require some sort of configuration file to operate, and need that configuration to by dynamic over time. The intent of this class is to provide a common pattern for config generators, with solutions for common problems like monitoring, environment variables, and signal handling.

To use this class for your own config generator, you need to:

  1. Subclass this class.

  2. Declare all the environment variables you care about, with the ServiceSkeleton declaration methods string, integer, etc.

  3. Implement service-specific config generation and reloading code, by overriding the private methods #config_file, #config_data, and #reload_server (and also potentially #config_ok?, #sleep_duration, #before_regenerate_config, and #after_regenerate_config). See the documentation for those methods for what they need to do.

  4. Setup any file watchers you want with .watch and #watch.

  5. Use the ServiceSkeleton Runner to start the service. Something like this should do the trick:

    class MyConfigGenerator < ConfigSkeleton # Implement all the necessary methods end

    ServiceSkeleton::Runner.new(MyConfigGenerator, ENV).run if FILE == $0

  6. Sit back and relax.

Environment Variables

In keeping with the principles of the 12 factor app, all configuration of the config generator should generally be done via the process environment. To make this easier, ConfigSkeleton leverages ServiceSkeleton's configuration system to allow you to declare environment variables of various types, provide defaults, and access the configuration values via the config method. See the ServiceSkeleton documentation for more details on how all this is accomplished.

Signal Handling

Config generators automatically hook several signals when they are created:

  • SIGHUP: Trigger a regeneration of the config file and force a reload of the associated server.

  • SIGINT/SIGTERM: Immediately terminate the process.

  • SIGUSR1/SIGUSR2: Increase (USR1) or decrease (USR2) the verbosity of log messages output.

Exported Metrics

No modern system is complete without Prometheus metrics. These can be scraped by making a HTTP request to the /metrics path on the port specified by the <SERVICEPREFIX>_METRICS_PORT environment variable (if no port is specified, the metrics server is turned off, for security). The metrics server will provide the config generator-specific metrics by default:

  • **`_generation_requests_total: The number of times the config generator has tried to generate a new config. This includes any attempts that failed due to exception.

  • <prefix>_generation_request_duration_seconds{,_sum,_count}: A histogram of the amount of time taken for the config_data method to generate a new config.

  • <prefix>_generation_exceptions_total: A set of counters which record the number of times the config_data method raised an exception, labelled with the class of the exception that occurred. The backtrace and error message should also be present in the logs.

  • ** <prefix>_generation_in_progress_count**: A gauge that should be either 1 or 0, depending on whether the config_data method is currently being called.

  • <prefix>_last_generation_timestamp: A floating-point number of seconds since the Unix epoch indicating when a config was last successfully generated. This timestamp is updated every time the config generator checks to see if the config has changed, whether or not a new config is written.

  • <prefix>_last_change_timestamp: A floating-point number of seconds since the Unix epoch indicating when the config was last changed (that is, a new config file written and the server reloaded).

  • <prefix>_reload_total: A set of counters indicating the number of times the server has been asked to reload, usually as the result of a changed config file, but potentially also due to receiving a SIGHUP. The counters are labelled by the status of the reload: "success" (all is well), "failure" (the attempt to reload failed, indicating a problem in the reload_server method), "bad-config" (the server reload succeeded, but the config_ok? check subsequently failed), or "everything-is-awful" (the config_ok? check failed both before and after the reload, indicating something is very wrong with the underlying server).

  • <prefix>_signals_total: A set of counters indicating how many of each signal have been received, labelled by signal.

  • <prefix>_config_ok: A gauge that should be either 1 or 0, depending on whether the last generated config was loaded successfully by the server. If the #config_ok? method has not been overridden, this will always be 1.

Note that all of the above metrics have a <prefix> at the beginning; the value of this is derived from the class name, by snake-casing.

Watching files

Sometimes your config, or the server, relies on other files on the filesystem managed by other systems (typically a configuration management system), and when those change, the config needs to change, or the server needs to be reloaded. To accommodate this requirement, you can declare a "file watch" in your config generator, and any time the file or directory being watched changes, a config regeneration and server reload will be forced.

To declare a file watch, just call the .watch class method, or #watch instance method, passing one or more strings containing the full path to files or directories to watch.

Defined Under Namespace

Classes: ConfigRegenNotifier, Error, NotImplementedError

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(*_, metrics:, config:) ⇒ ConfigSkeleton

Returns a new instance of ConfigSkeleton.



214
215
216
217
218
219
220
221
222
223
 
# File 'lib/config_skeleton.rb', line 214

def initialize(*_, metrics:, config:)
  super
  initialize_config_skeleton_metrics
  @trigger_regen_r, @trigger_regen_w = IO.pipe
  @terminate_r, @terminate_w = IO.pipe

  raise "cooldown_duration invalid" if cooldown_duration < 0
  raise "sleep_duration invalid" if sleep_duration < 0
  raise "sleep_duration must not be less than cooldown_duration" if sleep_duration < cooldown_duration
end

Class Method Details

.inherited(klass) ⇒ Object 



167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
 
# File 'lib/config_skeleton.rb', line 167

def self.inherited(klass)
  klass.boolean "#{klass.service_name.upcase}_CONFIG_ONESHOT".to_sym, default: false

  klass.gauge :"#{klass.service_name}_config_ok", docstring: "Whether the last config change was accepted by the server"
  klass.gauge :"#{klass.service_name}_generation_ok", docstring: "Whether the last config generation completed without error"
  klass.gauge :"#{klass.service_name}_last_generation_timestamp", docstring: "When the last config generation run was made"
  klass.gauge :"#{klass.service_name}_last_change_timestamp", docstring: "When the config file was last written to"
  klass.counter :"#{klass.service_name}_reload_total", docstring: "How many times we've asked the server to reload", labels: [:status]
  klass.counter :"#{klass.service_name}_signals_total", docstring: "How many signals have been received (and handled)"

  klass.hook_signal("HUP") do
    logger.info("SIGHUP") { "received SIGHUP, triggering config regeneration" }
    @trigger_regen_w << "."
  end
end

.watch(*f) ⇒ void

This method returns an undefined value.

Declare a file watch on all instances of the config generator.

When you're looking to watch a file whose path is well-known and never-changing, you can declare the watch in the class.

Examples:

reload every time a logfile is written to

class MyConfig
  watch "/var/log/syslog"
end

Parameters:

  • f (String) —

    one or more file paths to watch.

See Also:



199
200
201
202
 
# File 'lib/config_skeleton.rb', line 199

def self.watch(*f)
  @watches ||= []
  @watches += f
end

.watches ⇒ Array<String>

Retrieve the list of class-level file watches.

Not interesting for most users.

Returns:

  • (Array<String>) 


210
211
212
 
# File 'lib/config_skeleton.rb', line 210

def self.watches
  @watches || []
end

Instance Method Details

#regen_notifier ⇒ ConfigRegenNotifier

Expose the write pipe which can be written to to trigger a config regeneration with a forced reload; a similar mechanism is used for shutdown but in that case writes are managed internally.

Usage: config.regen_notifier.trigger_regen

Returns:



232
233
234
 
# File 'lib/config_skeleton.rb', line 232

def regen_notifier
  @regen_notifier ||= ConfigRegenNotifier.new(@trigger_regen_w)
end

#run ⇒ void

This method returns an undefined value.

Set the config generator running.

Does the needful to generate configs and reload the server. Typically never returns, unless you send the process a SIGTERM/SIGINT.



243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
 
# File 'lib/config_skeleton.rb', line 243

def run
  logger.info(logloc) { "Commencing config management" }

  write_initial_config

  if config.config_oneshot
    logger.info(logloc) { "Oneshot run specified - exiting" }
    Process.kill("TERM", $PID)
  end

  watch(*self.class.watches)

  logger.debug(logloc) { "notifier fd is #{notifier.to_io.inspect}" }

  loop do
    if cooldown_duration > 0
      logger.debug(logloc) { "Sleeping for #{cooldown_duration} seconds (cooldown)" }
      IO.select([@terminate_r], [], [], cooldown_duration)
    end

    timeout = sleep_duration - cooldown_duration
    logger.debug(logloc) { "Sleeping for #{timeout} seconds unless interrupted" }
    ios = IO.select([notifier.to_io, @terminate_r, @trigger_regen_r], [], [], timeout)

    if ios
      if ios.first.include?(notifier.to_io)
        logger.debug(logloc) { "inotify triggered" }
        notifier.process
        regenerate_config(force_reload: true)
      elsif ios.first.include?(@terminate_r)
        logger.debug(logloc) { "triggered by termination pipe" }
        break
      elsif ios.first.include?(@trigger_regen_r)
        # we want to wait until everything in the backlog is read
        # before proceeding so we don't run out of buffer memory
        # for the pipe
        while @trigger_regen_r.read_nonblock(20, nil, exception: false) != :wait_readable; end

        logger.debug(logloc) { "triggered by regen pipe" }
        regenerate_config(force_reload: true)
      else
        logger.error(logloc) { "Mysterious return from select: #{ios.inspect}" }
      end
    else
      logger.debug(logloc) { "triggered by timeout" }
      regenerate_config
    end
  end
end

#shutdown ⇒ Object

Trigger the run loop to stop running.



295
296
297
 
# File 'lib/config_skeleton.rb', line 295

def shutdown
  @terminate_w.write(".")
end

#watch(*files) ⇒ void

This method returns an undefined value.

Setup a file watch.

If the files you want to watch could be in different places on different systems (for instance, if your config generator's working directory can be configured via environment), then you'll need to call this in your class' initialize method to setup the watch.

Watching a file, for our purposes, simply means that whenever it is modified, the config is regenerated and the server process reloaded.

Watches come in two flavours: file watches, and directory watches. A file watch is straightforward: if the contents of the file are modified, off we go. For a directory, if a file is created in the directory, or deleted from the directory, or if any file in the directory is modified, the regen/reload process is triggered. Note that directory watches are recursive; all files and subdirectories under the directory specified will be watched.

Parameters:

  • files (Array<String>) —

    the paths to watch for changes.

See Also:



323
324
325
326
327
328
329
330
331
332
 
# File 'lib/config_skeleton.rb', line 323

def watch(*files)
  return if ENV["DISABLE_INOTIFY"]
  files.each do |f|
    if File.directory?(f)
      notifier.watch(f, :recursive, :create, :modify, :delete, :move) { |ev| logger.info("#{logloc} watcher") { "detected #{ev.flags.join(", ")} on #{ev.watcher.path}/#{ev.name}; regenerating config" } }
    else
      notifier.watch(f, :close_write) { |ev| logger.info("#{logloc} watcher") { "detected #{ev.flags.join(", ")} on #{ev.watcher.path}; regenerating config" } }
    end
  end
end