Module: Raix::ChatCompletion

Extended by:
ActiveSupport::Concern
Included in:
PromptDeclarations
Defined in:
lib/raix/chat_completion.rb

Overview

The ‘ChatCompletion“ module is a Rails concern that provides a way to interact with the OpenRouter Chat Completion API via its client. The module includes a few methods that allow you to build a transcript of messages and then send them to the API for completion. The API will return a response that you can use however you see fit. If the response includes a function call, the module will dispatch the function call and return the result. Which implies that function calls need to be defined on the class that includes this module. (Note: You should probably use the `FunctionDispatch` module to define functions instead of doing it manually.)

Instance Attribute Summary collapse

Instance Method Summary collapse

Instance Attribute Details

#frequency_penalty ⇒ Object

Returns the value of attribute frequency_penalty.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def frequency_penalty
  @frequency_penalty
end

#logit_bias ⇒ Object

Returns the value of attribute logit_bias.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def logit_bias
  @logit_bias
end

#logprobs ⇒ Object

Returns the value of attribute logprobs.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def logprobs
  @logprobs
end

#loop ⇒ Object

Returns the value of attribute loop.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def loop
  @loop
end

#max_tokens ⇒ Object

Returns the value of attribute max_tokens.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def max_tokens
  @max_tokens
end

#min_p ⇒ Object

Returns the value of attribute min_p.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def min_p
  @min_p
end

#model ⇒ Object

Returns the value of attribute model.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def model
  @model
end

#presence_penalty ⇒ Object

Returns the value of attribute presence_penalty.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def presence_penalty
  @presence_penalty
end

#provider ⇒ Object

Returns the value of attribute provider.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def provider
  @provider
end

#repetition_penalty ⇒ Object

Returns the value of attribute repetition_penalty.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def repetition_penalty
  @repetition_penalty
end

#response_format ⇒ Object

Returns the value of attribute response_format.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def response_format
  @response_format
end

#seed ⇒ Object

Returns the value of attribute seed.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def seed
  @seed
end

#stop ⇒ Object

Returns the value of attribute stop.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def stop
  @stop
end

#stream ⇒ Object

Returns the value of attribute stream.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def stream
  @stream
end

#temperature ⇒ Object

Returns the value of attribute temperature.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def temperature
  @temperature
end

#tool_choice ⇒ Object

Returns the value of attribute tool_choice.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def tool_choice
  @tool_choice
end

#tools ⇒ Object

Returns the value of attribute tools.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def tools
  @tools
end

#top_a ⇒ Object

Returns the value of attribute top_a.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def top_a
  @top_a
end

#top_k ⇒ Object

Returns the value of attribute top_k.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def top_k
  @top_k
end

#top_logprobs ⇒ Object

Returns the value of attribute top_logprobs.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def top_logprobs
  @top_logprobs
end

#top_p ⇒ Object

Returns the value of attribute top_p.



20
21
22
 
# File 'lib/raix/chat_completion.rb', line 20

def top_p
  @top_p
end

Instance Method Details

#chat_completion(params: {}, loop: false, json: false, raw: false, openai: false) ⇒ String|Hash

This method performs chat completion based on the provided transcript and parameters.

Parameters:

  • params (Hash) (defaults to: {}) —

    The parameters for chat completion.

  • loop (Hash) (defaults to: false) —

    a customizable set of options

Options Hash (params:):

  • :json (Boolean) — default: false —

    Whether to return the parse the response as a JSON object.

  • :openai (Boolean) — default: false —

    Whether to use OpenAI’s API instead of OpenRouter’s.

  • :raw (Boolean) — default: false —

    Whether to return the raw response or dig the text content.

Options Hash (loop:):

  • :loop (Boolean) — default: false —

    Whether to loop the chat completion after function calls.

Returns:

  • (String|Hash) —

    The completed chat response.



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/raix/chat_completion.rb', line 32

def chat_completion(params: {}, loop: false, json: false, raw: false, openai: false)
  messages = transcript.flatten.compact.map { |msg| transform_message_format(msg) }
  raise "Can't complete an empty transcript" if messages.blank?

  # used by FunctionDispatch
  self.loop = loop

  # set params to default values if not provided
  params[:frequency_penalty] ||= frequency_penalty.presence
  params[:logit_bias] ||= logit_bias.presence
  params[:logprobs] ||= logprobs.presence
  params[:max_tokens] ||= max_tokens.presence || Raix.configuration.max_tokens
  params[:min_p] ||= min_p.presence
  params[:presence_penalty] ||= presence_penalty.presence
  params[:provider] ||= provider.presence
  params[:repetition_penalty] ||= repetition_penalty.presence
  params[:response_format] ||= response_format.presence
  params[:seed] ||= seed.presence
  params[:stop] ||= stop.presence
  params[:temperature] ||= temperature.presence || Raix.configuration.temperature
  params[:tool_choice] ||= tool_choice.presence
  params[:tools] ||= tools.presence
  params[:top_a] ||= top_a.presence
  params[:top_k] ||= top_k.presence
  params[:top_logprobs] ||= top_logprobs.presence
  params[:top_p] ||= top_p.presence

  if json
    params[:provider] ||= {}
    params[:provider][:require_parameters] = true
    params[:response_format] ||= {}
    params[:response_format][:type] = "json_object"
  end

  # set the model to the default if not provided
  self.model ||= Raix.configuration.model

  begin
    response = if openai
                 openai_request(params:, model: openai,
                                messages:)
               else
                 openrouter_request(
                   params:, model:, messages:
                 )
               end
    retry_count = 0
    content = nil

    # no need for additional processing if streaming
    return if stream && response.blank?

    # tuck the full response into a thread local in case needed
    Thread.current[:chat_completion_response] = response.with_indifferent_access

    # TODO: add a standardized callback hook for usage events
    # broadcast(:usage_event, usage_subject, self.class.name.to_s, response, premium?)

    # TODO: handle parallel tool calls
    if (function = response.dig("choices", 0, "message", "tool_calls", 0, "function"))
      @current_function = function["name"]
      # dispatch the called function
      arguments = JSON.parse(function["arguments"].presence || "{}")
      arguments[:bot_message] = bot_message if respond_to?(:bot_message)
      return send(function["name"], arguments.with_indifferent_access)
    end

    response.tap do |res|
      content = res.dig("choices", 0, "message", "content")
      if json
        content = content.squish
        return JSON.parse(content)
      end

      return content unless raw
    end
  rescue JSON::ParserError => e
    if e.message.include?("not a valid") # blank JSON
      puts "Retrying blank JSON response... (#{retry_count} attempts) #{e.message}"
      retry_count += 1
      sleep 1 * retry_count # backoff
      retry if retry_count < 3

      raise e # just fail if we can't get content after 3 attempts
    end

    # attempt to fix the JSON
    JsonFixer.new.call(content, e.message)
  rescue Faraday::BadRequestError => e
    # make sure we see the actual error message on console or Honeybadger
    puts "Chat completion failed!!!!!!!!!!!!!!!!: #{e.response[:body]}"
    raise e
  end
end

#transcript ⇒ Array

This method returns the transcript array. Manually add your messages to it in the following abbreviated format before calling ‘chat_completion`.

{ system: “You are a pumpkin” }, { user: “Hey what time is it?” }, { assistant: “Sorry, pumpkins do not wear watches” }

to add a function result use the following format: { function: result, name: ‘fancy_pants_function’ }

Returns:

  • (Array) —

    The transcript array.



139
140
141
 
# File 'lib/raix/chat_completion.rb', line 139

def transcript
  @transcript ||= []
end