Module: Tracebook

Defined in:
lib/tracebook.rb,
lib/tracebook/config.rb,
lib/tracebook/engine.rb,
lib/tracebook/errors.rb,
lib/tracebook/result.rb,
lib/tracebook/mappers.rb,
lib/tracebook/version.rb,
lib/tracebook/mappers/base.rb,
lib/tracebook/mappers/ollama.rb,
lib/tracebook/mappers/openai.rb,
lib/tracebook/redactors/base.rb,
lib/tracebook/redactors/email.rb,
lib/tracebook/redactors/phone.rb,
lib/tracebook/adapters/ruby_llm.rb,
lib/tracebook/mappers/anthropic.rb,
lib/tracebook/pricing/calculator.rb,
lib/tracebook/redaction_pipeline.rb,
lib/tracebook/redactors/card_pan.rb,
lib/tracebook/adapters/active_agent.rb,
lib/tracebook/normalized_interaction.rb,
lib/generators/tracebook/install/install_generator.rb,
app/jobs/tracebook/export_job.rb,
app/models/tracebook/interaction.rb,
app/models/tracebook/pricing_rule.rb,
app/models/tracebook/rollup_daily.rb,
app/jobs/tracebook/application_job.rb,
app/models/tracebook/redaction_rule.rb,
app/jobs/tracebook/daily_rollups_job.rb,
app/models/tracebook/application_record.rb,
app/helpers/tracebook/application_helper.rb,
app/mailers/tracebook/application_mailer.rb,
app/helpers/tracebook/interactions_helper.rb,
app/jobs/tracebook/persist_interaction_job.rb,
app/controllers/tracebook/exports_controller.rb,
app/controllers/tracebook/application_controller.rb,
app/controllers/tracebook/interactions_controller.rb

Overview

TraceBook is a Rails engine for capturing, storing, and reviewing LLM interactions.

It provides:

  • Automatic redaction of PII from request/response payloads
  • Encrypted storage of sensitive data using ActiveRecord::Encryption
  • Cost tracking based on token usage and configurable pricing rules
  • Review workflow (approve/flag/reject) with audit trail
  • Hotwire-powered dashboard UI with filtering and export
  • Built-in adapters for OpenAI, Anthropic, Ollama
  • Support for hierarchical agent sessions (parent-child relationships)

Examples:

Basic configuration

TraceBook.configure do |config|
  config.authorize = ->(user, action, resource) { user&.admin? }
  config.project_name = "My App"
  config.persist_async = Rails.env.production?
end

Recording an interaction

TraceBook.record!(
  provider: "openai",
  model: "gpt-4o",
  request_payload: { messages: messages },
  response_payload: response,
  input_tokens: 100,
  output_tokens: 50,
  user: current_user,
  tags: ["production", "support"]
)

See Also:

Defined Under Namespace

Modules: Adapters, ApplicationHelper, Generators, InteractionsHelper, Mappers, Pricing, Redactors Classes: ApplicationController, ApplicationJob, ApplicationMailer, ApplicationRecord, Config, ConfigurationError, DailyRollupsJob, Engine, Error, ExportJob, ExportsController, Interaction, InteractionsController, NormalizedInteraction, PersistInteractionJob, PricingRule, RedactionPipeline, RedactionRule, Result, RollupDaily

Constant Summary collapse

VERSION = 
"0.1.1"

Class Method Summary collapse

Class Method Details

.build_normalized_interaction(attributes) ⇒ Object (private) 



177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
 
# File 'lib/tracebook.rb', line 177

def build_normalized_interaction(attributes)
  NormalizedInteraction.new(
    provider: attributes.fetch(:provider),
    model: attributes.fetch(:model),
    project: attributes[:project],
    request_payload: attributes[:request_payload],
    response_payload: attributes[:response_payload],
    request_text: attributes[:request_text],
    response_text: attributes[:response_text],
    input_tokens: attributes[:input_tokens],
    output_tokens: attributes[:output_tokens],
    latency_ms: attributes[:latency_ms],
    status: attributes.fetch(:status, "success"),
    error_class: attributes[:error_class],
    error_message: attributes[:error_message],
    tags: Array(attributes[:tags]).compact,
    metadata: attributes[:metadata] || {},
    user: attributes[:user],
    parent_id: attributes[:parent_id],
    session_id: attributes[:session_id]
  )
end

.config ⇒ Tracebook::Config

Returns the current configuration instance.

Returns:

  • the configuration object



50
51
52
 
# File 'lib/tracebook.rb', line 50

def config
  @config ||= Config.new
end

.configure {|config| ... } ⇒ Tracebook::Config

Configures TraceBook with a block.

Configuration is frozen after the block executes. Call #reset_configuration! in tests to reset.

Examples:

TraceBook.configure do |config|
  config.authorize = ->(user, action, resource) { user&.admin? }
  config.persist_async = true
  config.project_name = "Support App"
end

Yields:

  • (config) —

    Yields the config object for modification

Yield Parameters:

Returns:

  • the finalized configuration

Raises:

  • if configuration is already finalized



70
71
72
73
74
75
76
 
# File 'lib/tracebook.rb', line 70

def configure
  ensure_configurable!

  yield(config)
  finalize_configuration!
  config
end

.ensure_configurable! ⇒ Object (private)

Raises:



171
172
173
174
175
 
# File 'lib/tracebook.rb', line 171

def ensure_configurable!
  return unless @configuration_finalized || config.finalized?

  raise ConfigurationError, "TraceBook configuration is already finalized"
end

.finalize_configuration! ⇒ Object (private) 



166
167
168
169
 
# File 'lib/tracebook.rb', line 166

def finalize_configuration!
  config.finalize!
  @configuration_finalized = true
end

.record!(**attributes) ⇒ Tracebook::Result

Records an LLM interaction.

When config.persist_async is true, the interaction is enqueued via PersistInteractionJob. Otherwise, it's persisted inline.

Examples:

Recording a successful completion

result = TraceBook.record!(
  provider: "openai",
  model: "gpt-4o-mini",
  request_payload: { messages: [{ role: "user", content: "Hello" }] },
  response_payload: { choices: [{ message: { content: "Hi!" } }] },
  input_tokens: 10,
  output_tokens: 5,
  latency_ms: 150,
  status: :success,
  user: current_user,
  tags: ["greeting"]
)

Recording a failed request

TraceBook.record!(
  provider: "anthropic",
  model: "claude-3-5-sonnet",
  request_payload: request,
  response_payload: nil,
  status: :error,
  error_class: "Faraday::TimeoutError",
  error_message: "Request timed out after 30s",
  latency_ms: 30000
)

Parameters:

  • Interaction attributes

Options Hash (**attributes):

  • :provider (String) —

    Provider name (e.g., "openai", "anthropic") required

  • :model (String) —

    Model identifier (e.g., "gpt-4o", "claude-3-5-sonnet") required

  • :project (String, nil) —

    Project name for filtering

  • :request_payload (Hash, nil) —

    Full request sent to provider (will be encrypted)

  • :response_payload (Hash, nil) —

    Full response from provider (will be encrypted)

  • :request_text (String, nil) —

    Human-readable request summary

  • :response_text (String, nil) —

    Human-readable response summary

  • :input_tokens (Integer, nil) —

    Prompt token count

  • :output_tokens (Integer, nil) —

    Completion token count

  • :latency_ms (Integer, nil) —

    Request duration in milliseconds

  • :status (Symbol, String) —

    :success, :error, or :canceled (default: :success)

  • :error_class (String, nil) —

    Exception class name on failure

  • :error_message (String, nil) —

    Exception message on failure

  • :tags (Array<String>) —

    Labels for filtering (e.g., ["prod", "urgent"])

  • :metadata (Hash) —

    Custom metadata (e.g., { ticket_id: 123 })

  • :user (ActiveRecord::Base, nil) —

    Associated user (polymorphic)

  • :session_id (String, nil) —

    Session identifier for grouping related calls

  • :parent_id (Integer, nil) —

    Parent interaction ID for hierarchical chains

  • :idempotency_key (String, nil) —

    Key for deduplication

Returns:

  • Result object with success/error information



149
150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/tracebook.rb', line 149

def record!(**attributes)
  payload = build_normalized_interaction(attributes)
  result = Result.new(idempotency_key: attributes[:idempotency_key])

  if config.persist_async
    PersistInteractionJob.perform_later(payload.to_h)
    result
  else
    interaction = PersistInteractionJob.perform_now(payload.to_h)
    Result.new(interaction: interaction, idempotency_key: attributes[:idempotency_key])
  end
rescue StandardError => error
  Result.new(error: error, idempotency_key: attributes[:idempotency_key])
end

.reset_configuration! ⇒ void

This method returns an undefined value.

Resets configuration to a clean state.

Used in tests to start with fresh configuration between test cases.

Examples:

In test setup

setup do
  TraceBook.reset_configuration!
  TraceBook.configure do |config|
    config.authorize = ->(*) { true }
  end
end



91
92
93
94
 
# File 'lib/tracebook.rb', line 91

def reset_configuration!
  @config = Config.new
  @configuration_finalized = false
end