Module: SecApi::MetricsCollector

Extended by:
MetricsCollector
Included in:
MetricsCollector
Defined in:
lib/sec_api/metrics_collector.rb

Overview

Provides centralized metric recording for SEC API operations.

This module standardizes metric names and tags across all integrations. Use directly with your metrics backend, or configure via metrics_backend for automatic metrics collection.

All metric methods are safe to call - they will not raise exceptions even if the backend is nil, misconfigured, or raises errors. This ensures that metrics never break production operations.

Examples:

Direct usage with StatsD

statsd = Datadog::Statsd.new('localhost', 8125)

config.on_response = ->(request_id:, status:, duration_ms:, url:, method:) {
  SecApi::MetricsCollector.record_response(statsd,
    status: status,
    duration_ms: duration_ms,
    method: method
  )
}

Automatic metrics with metrics_backend

config = SecApi::Config.new(
  api_key: "...",
  metrics_backend: Datadog::Statsd.new('localhost', 8125)
)
# Metrics automatically collected for all operations

New Relic custom events

config.on_response = ->(request_id:, status:, duration_ms:, url:, method:) {
  NewRelic::Agent.record_custom_event("SecApiRequest", {
    request_id: request_id,
    status: status,
    duration_ms: duration_ms,
    method: method.to_s.upcase
  })
}

Datadog APM integration

config.on_request = ->(request_id:, method:, url:, headers:) {
  Datadog::Tracing.trace('sec_api.request') do |span|
    span.set_tag('request_id', request_id)
    span.set_tag('http.method', method.to_s.upcase)
    span.set_tag('http.url', url)
  end
}

OpenTelemetry spans

tracer = OpenTelemetry.tracer_provider.tracer('sec_api')

config.on_request = ->(request_id:, method:, url:, headers:) {
  tracer.in_span('sec_api.request', attributes: {
    'sec_api.request_id' => request_id,
    'http.method' => method.to_s.upcase,
    'http.url' => url
  }) { }
}

Prometheus push gateway

prometheus = Prometheus::Client.registry
requests_total = prometheus.counter(:sec_api_requests_total, labels: [:method, :status])
duration_histogram = prometheus.histogram(:sec_api_request_duration_seconds, labels: [:method])

config.on_response = ->(request_id:, status:, duration_ms:, url:, method:) {
  requests_total.increment(labels: {method: method.to_s.upcase, status: status.to_s})
  duration_histogram.observe(duration_ms / 1000.0, labels: {method: method.to_s.upcase})
}

Constant Summary collapse

REQUESTS_TOTAL =

Metric name for total requests made (counter).

Returns:

  • (String) —

    metric name

"sec_api.requests.total"
REQUESTS_SUCCESS =

Metric name for successful requests (counter, status < 400).

Returns:

  • (String) —

    metric name

"sec_api.requests.success"
REQUESTS_ERROR =

Metric name for error requests (counter, status >= 400).

Returns:

  • (String) —

    metric name

"sec_api.requests.error"
REQUESTS_DURATION =

Metric name for request duration (histogram, milliseconds).

Returns:

  • (String) —

    metric name

"sec_api.requests.duration_ms"
RETRIES_TOTAL =

Metric name for retry attempts (counter).

Returns:

  • (String) —

    metric name

"sec_api.retries.total"
RETRIES_EXHAUSTED =

Metric name for exhausted retries (counter).

Returns:

  • (String) —

    metric name

"sec_api.retries.exhausted"
RATE_LIMIT_HIT =

Metric name for rate limit hits (counter, 429 responses).

Returns:

  • (String) —

    metric name

"sec_api.rate_limit.hit"
RATE_LIMIT_THROTTLE =

Metric name for proactive throttling events (counter).

Returns:

  • (String) —

    metric name

"sec_api.rate_limit.throttle"
STREAM_FILINGS =

Metric name for streaming filings received (counter).

Returns:

  • (String) —

    metric name

"sec_api.stream.filings"
STREAM_LATENCY =

Metric name for streaming delivery latency (histogram, milliseconds).

Returns:

  • (String) —

    metric name

"sec_api.stream.latency_ms"
STREAM_RECONNECTS =

Metric name for stream reconnection events (counter).

Returns:

  • (String) —

    metric name

"sec_api.stream.reconnects"
JOURNEY_STAGE_DURATION =

Metric name for filing journey stage duration (histogram, milliseconds).

Returns:

  • (String) —

    metric name

"sec_api.filing.journey.stage_ms"
JOURNEY_TOTAL_DURATION =

Metric name for total filing journey duration (histogram, milliseconds).

Returns:

  • (String) —

    metric name

"sec_api.filing.journey.total_ms"

Instance Method Summary collapse

Instance Method Details

#record_error(backend, error_class:, method:) ⇒ void

This method returns an undefined value.

Records a final error (all retries exhausted).

Examples:

Record a final error

MetricsCollector.record_error(statsd, error_class: "SecApi::NetworkError", method: :get)

Parameters:

  • backend (Object) —

    Metrics backend

  • error_class (String) —

    Exception class name

  • method (Symbol) —

    HTTP method



183
184
185
186
 
# File 'lib/sec_api/metrics_collector.rb', line 183

def record_error(backend, error_class:, method:)
  tags = {error_class: error_class, method: method.to_s.upcase}
  increment(backend, RETRIES_EXHAUSTED, tags: tags)
end

#record_filing(backend, latency_ms:, form_type:) ⇒ void

This method returns an undefined value.

Records a streaming filing received.

Examples:

Record a filing receipt

MetricsCollector.record_filing(statsd, latency_ms: 500, form_type: "10-K")

Parameters:

  • backend (Object) —

    Metrics backend

  • latency_ms (Integer) —

    Filing delivery latency in milliseconds

  • form_type (String) —

    Filing form type (10-K, 8-K, etc.)



231
232
233
234
235
 
# File 'lib/sec_api/metrics_collector.rb', line 231

def record_filing(backend, latency_ms:, form_type:)
  tags = {form_type: form_type}
  increment(backend, STREAM_FILINGS, tags: tags)
  histogram(backend, STREAM_LATENCY, latency_ms, tags: tags)
end

#record_journey_stage(backend, stage:, duration_ms:, form_type: nil) ⇒ void

This method returns an undefined value.

Records a filing journey stage completion.

Use this to track duration of individual pipeline stages (detected, queried, extracted, processed). Combined with FilingJourney logging, this provides both detailed logs and aggregated metrics.

Examples:

Record a query stage

MetricsCollector.record_journey_stage(statsd,
  stage: "queried",
  duration_ms: 150,
  form_type: "10-K"
)

Parameters:

  • backend (Object) —

    Metrics backend

  • stage (String) —

    Journey stage (detected, queried, extracted, processed)

  • duration_ms (Integer) —

    Stage duration in milliseconds

  • form_type (String, nil) (defaults to: nil) —

    Filing form type (10-K, 8-K, etc.)

See Also:



274
275
276
277
278
 
# File 'lib/sec_api/metrics_collector.rb', line 274

def record_journey_stage(backend, stage:, duration_ms:, form_type: nil)
  tags = {stage: stage}
  tags[:form_type] = form_type if form_type
  histogram(backend, JOURNEY_STAGE_DURATION, duration_ms, tags: tags)
end

#record_journey_total(backend, total_ms:, form_type: nil, success: true) ⇒ void

This method returns an undefined value.

Records total filing journey duration.

Use this to track end-to-end pipeline latency from filing detection through processing completion. Useful for monitoring SLAs and identifying slow pipelines.

Examples:

Record successful pipeline

MetricsCollector.record_journey_total(statsd,
  total_ms: 5000,
  form_type: "10-K",
  success: true
)

Record failed pipeline

MetricsCollector.record_journey_total(statsd,
  total_ms: 500,
  form_type: "10-K",
  success: false
)

Parameters:

  • backend (Object) —

    Metrics backend

  • total_ms (Integer) —

    Total pipeline duration in milliseconds

  • form_type (String, nil) (defaults to: nil) —

    Filing form type (10-K, 8-K, etc.)

  • success (Boolean) (defaults to: true) —

    Whether processing succeeded (default: true)

See Also:



308
309
310
311
312
 
# File 'lib/sec_api/metrics_collector.rb', line 308

def record_journey_total(backend, total_ms:, form_type: nil, success: true)
  tags = {success: success.to_s}
  tags[:form_type] = form_type if form_type
  histogram(backend, JOURNEY_TOTAL_DURATION, total_ms, tags: tags)
end

#record_rate_limit(backend, retry_after: nil) ⇒ void

This method returns an undefined value.

Records a rate limit (429) response.

Examples:

Record a rate limit hit

MetricsCollector.record_rate_limit(statsd, retry_after: 30)

Parameters:

  • backend (Object) —

    Metrics backend

  • retry_after (Integer, nil) (defaults to: nil) —

    Seconds to wait before retry



197
198
199
200
 
# File 'lib/sec_api/metrics_collector.rb', line 197

def record_rate_limit(backend, retry_after: nil)
  increment(backend, RATE_LIMIT_HIT)
  gauge(backend, "sec_api.rate_limit.retry_after", retry_after) if retry_after
end

#record_reconnect(backend, attempt_count:, downtime_seconds:) ⇒ void

This method returns an undefined value.

Records a stream reconnection.

Examples:

Record a reconnection

MetricsCollector.record_reconnect(statsd, attempt_count: 3, downtime_seconds: 15.5)

Parameters:

  • backend (Object) —

    Metrics backend

  • attempt_count (Integer) —

    Number of reconnection attempts

  • downtime_seconds (Float) —

    Total downtime in seconds



247
248
249
250
251
 
# File 'lib/sec_api/metrics_collector.rb', line 247

def record_reconnect(backend, attempt_count:, downtime_seconds:)
  increment(backend, STREAM_RECONNECTS)
  gauge(backend, "sec_api.stream.reconnect_attempts", attempt_count)
  histogram(backend, "sec_api.stream.downtime_ms", (downtime_seconds * 1000).round)
end

#record_response(backend, status:, duration_ms:, method:) ⇒ void

This method returns an undefined value.

Records a successful or failed response.

Increments request counters and records duration histogram. Status codes < 400 are considered successful, >= 400 are errors.

Examples:

Record a successful response

MetricsCollector.record_response(statsd, status: 200, duration_ms: 150, method: :get)

Record an error response

MetricsCollector.record_response(statsd, status: 429, duration_ms: 50, method: :get)

Parameters:

  • backend (Object) —

    Metrics backend (StatsD, Datadog::Statsd, etc.)

  • status (Integer) —

    HTTP status code

  • duration_ms (Integer) —

    Request duration in milliseconds

  • method (Symbol) —

    HTTP method (:get, :post, etc.)



144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/sec_api/metrics_collector.rb', line 144

def record_response(backend, status:, duration_ms:, method:)
  tags = {method: method.to_s.upcase, status: status.to_s}

  increment(backend, REQUESTS_TOTAL, tags: tags)

  if status < 400
    increment(backend, REQUESTS_SUCCESS, tags: tags)
  else
    increment(backend, REQUESTS_ERROR, tags: tags)
  end

  histogram(backend, REQUESTS_DURATION, duration_ms, tags: tags)
end

#record_retry(backend, attempt:, error_class:) ⇒ void

This method returns an undefined value.

Records a retry attempt.

Examples:

Record a retry attempt

MetricsCollector.record_retry(statsd, attempt: 1, error_class: "SecApi::NetworkError")

Parameters:

  • backend (Object) —

    Metrics backend

  • attempt (Integer) —

    Retry attempt number (1-indexed)

  • error_class (String) —

    Exception class name that triggered retry



168
169
170
171
 
# File 'lib/sec_api/metrics_collector.rb', line 168

def record_retry(backend, attempt:, error_class:)
  tags = {attempt: attempt.to_s, error_class: error_class}
  increment(backend, RETRIES_TOTAL, tags: tags)
end

#record_throttle(backend, remaining:, delay:) ⇒ void

This method returns an undefined value.

Records proactive throttling.

Called when the rate limiter proactively delays a request to avoid hitting the rate limit.

Examples:

Record proactive throttling

MetricsCollector.record_throttle(statsd, remaining: 5, delay: 1.5)

Parameters:

  • backend (Object) —

    Metrics backend

  • remaining (Integer) —

    Requests remaining before limit

  • delay (Float) —

    Seconds the request was delayed



215
216
217
218
219
 
# File 'lib/sec_api/metrics_collector.rb', line 215

def record_throttle(backend, remaining:, delay:)
  increment(backend, RATE_LIMIT_THROTTLE)
  gauge(backend, "sec_api.rate_limit.remaining", remaining)
  histogram(backend, "sec_api.rate_limit.delay_ms", (delay * 1000).round)
end