Class: LLM::Tracer::Telemetry

Inherits:
LLM::Tracer show all
Defined in:
lib/llm/tracer/telemetry.rb

Overview

The LLM::Tracer::Telemetry tracer provides telemetry support through the opentelemetry-ruby RubyGem. The gem should be installed separately since this feature is opt-in and disabled by default. This feature exists to support integration with tools like LangSmith.

Examples:

InMemory export

#!/usr/bin/env ruby
require "llm"
require "pp"

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"
ses.tracer.spans.each { |span| pp span }

OTLP export

#!/usr/bin/env ruby
require "llm"
require "opentelemetry-exporter-otlp"

endpoint = "https://api.smith.langchain.com/otel/v1/traces"
exporter = OpenTelemetry::Exporter::OTLP::Exporter.new(endpoint:)

llm = LLM.openai(key: ENV["KEY"])
llm.tracer = LLM::Tracer::Telemetry.new(llm, exporter:)

ses = LLM::Session.new(llm)
ses.talk "hello"
ses.talk "how are you?"

See Also:

Instance Method Summary collapse

Methods inherited from LLM::Tracer

#inspect

Constructor Details

#initialize(provider, options = {}) ⇒ LLM::Tracer::Telemetry

param [LLM::Provider] provider An LLM provider



48
49
50
51
52
 
# File 'lib/llm/tracer/telemetry.rb', line 48

def initialize(provider, options = {})
  super
  @exporter = options.delete(:exporter)
  setup!
end

Instance Method Details

#flush! ⇒ nil

Note:

Exports are batched in the background by default. Long-lived processes usually do not need to call this method. Short-lived scripts should call #flush! before exit to reduce the risk of losing spans that are still buffered.

Flushes queued telemetry to the configured exporter.

Returns:

  • (nil) 


189
190
191
192
 
# File 'lib/llm/tracer/telemetry.rb', line 189

def flush!
  @tracer_provider.force_flush
  nil
end

#on_request_error(ex:, span:) ⇒ Object

Parameters:



113
114
115
116
117
118
119
120
 
# File 'lib/llm/tracer/telemetry.rb', line 113

def on_request_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.request.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_request_finish(operation:, res:, model: nil, span: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil)
  • res (LLM::Response)
  • span (Object, nil) (defaults to: nil) 


102
103
104
105
106
107
108
109
 
# File 'lib/llm/tracer/telemetry.rb', line 102

def on_request_finish(operation:, res:, model: nil, span: nil)
  return nil unless span
  case operation
  when "chat" then finish_chat(operation:, model:, res:, span:)
  when "retrieval" then finish_retrieval(operation:, res:, span:)
  else nil
  end
end

#on_request_start(operation:, model: nil) ⇒ Object

Parameters:

  • operation (String)
  • model (String) (defaults to: nil) 


92
93
94
95
96
97
98
 
# File 'lib/llm/tracer/telemetry.rb', line 92

def on_request_start(operation:, model: nil)
  case operation
  when "chat" then start_chat(operation:, model:)
  when "retrieval" then start_retrieval(operation:)
  else nil
  end
end

#on_tool_error(ex:, span:)

This method returns an undefined value.

Parameters:

  • ex (Exception) —

    The raised error.

  • span (Object, nil) —

    The span/context object returned by #on_tool_start.



160
161
162
163
164
165
166
167
 
# File 'lib/llm/tracer/telemetry.rb', line 160

def on_tool_error(ex:, span:)
  return nil unless span
  attributes = {"error.type" => ex.class.to_s}.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.status = ::OpenTelemetry::Trace::Status.error(ex.message)
  span.tap(&:finish)
end

#on_tool_finish(result:, span:)

This method returns an undefined value.

Parameters:



145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/llm/tracer/telemetry.rb', line 145

def on_tool_finish(result:, span:)
  return nil unless span
  attributes = {
    "gen_ai.tool.call.id" => result.id,
    "gen_ai.tool.name" => result.name,
    "gen_ai.tool.call.result" => LLM.json.dump(result.value)
  }.compact
  attributes.each { span.set_attribute(_1, _2) }
  span.add_event("gen_ai.tool.finish")
  span.tap(&:finish)
end

#on_tool_start(id:, name:, arguments:, model:)

This method returns an undefined value.

Parameters:

  • id (String) —

    The tool call ID assigned by the model/provider

  • name (String) —

    The tool (function) name.

  • arguments (Hash) —

    The parsed tool arguments.

  • model (String) —

    The model name



125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/llm/tracer/telemetry.rb', line 125

def on_tool_start(id:, name:, arguments:, model:)
  attributes = {
    "gen_ai.operation.name" => "execute_tool",
    "gen_ai.request.model" => model,
    "gen_ai.tool.call.id" => id,
    "gen_ai.tool.name" => name,
    "gen_ai.tool.call.arguments" => LLM.json.dump(arguments),
    "gen_ai.provider.name" => provider_name,
    "server.address" => provider_host,
    "server.port" => provider_port
  }.compact
  span_name = ["execute_tool", name].compact.join(" ")
  span = create_span(span_name.empty? ? "gen_ai.tool" : span_name, attributes:)
  span.add_event("gen_ai.tool.start")
  span
end

#spans ⇒ Array<OpenTelemetry::SDK::Trace::SpanData>

Note:

This method returns an empty array for exporters that do not implement 'finished_spans' such as the OTLP exporter

Returns:

  • (Array<OpenTelemetry::SDK::Trace::SpanData>) 


175
176
177
178
179
 
# File 'lib/llm/tracer/telemetry.rb', line 175

def spans
  return [] unless @exporter.respond_to?(:finished_spans)
  flush!
  @exporter.finished_spans
end

#start_trace(trace_group_id: nil, name: "llm", attributes: {}) ⇒ self

When +trace_group_id+ is provided, it is converted to an OpenTelemetry trace_id (via a deterministic 16-byte hash) so all spans until #stop_trace share that trace_id and appear as one trace in OTLP/Langfuse.

Parameters:

  • trace_group_id (String, nil) (defaults to: nil) —

    Optional. When present, converted to a 16-byte trace_id so all spans created until #stop_trace are grouped in one trace.

  • name (String) (defaults to: "llm") —

    Name for the root span (e.g. "chatbot.turn").

  • attributes (Hash) (defaults to: {}) —

    OpenTelemetry attributes to set on the root span.

Returns:

  • (self) 


61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
# File 'lib/llm/tracer/telemetry.rb', line 61

def start_trace(trace_group_id: nil, name: "llm", attributes: {})
  return self if trace_group_id.to_s.empty?

  span_context = span_context_from_trace_group_id(trace_group_id.to_s)
  parent_ctx = ::OpenTelemetry::Trace.context_with_span(
    ::OpenTelemetry::Trace.non_recording_span(span_context)
  )
  attrs = attributes.compact
  attrs["llm.trace_group_id"] = trace_group_id.to_s
  root_span = @tracer.start_span(
    name,
    kind: :server,
    attributes: attrs,
    with_parent: parent_ctx
  )
  thread[thread_root_span_key] = root_span
  thread[thread_root_context_key] = ::OpenTelemetry::Trace.context_with_span(root_span)
  self
end

#stop_trace ⇒ self

Returns:

  • (self) 


83
84
85
86
87
88
 
# File 'lib/llm/tracer/telemetry.rb', line 83

def stop_trace
  thread[thread_root_span_key]&.finish
  thread[thread_root_span_key] = nil
  thread[thread_root_context_key] = nil
  self
end