Class: Boxcars::Anthropic

Inherits:
Engine
  • Object
show all
Defined in:
lib/boxcars/engine/anthropic.rb

Overview

A engine that uses OpenAI’s API.

Constant Summary collapse

DEFAULT_PARAMS =

The default parameters to use when asking the engine.

{
  model: "claude-3-5-sonnet-20240620",
  max_tokens: 4096,
  temperature: 0.1
}.freeze
DEFAULT_NAME =

the default name of the engine

"Anthropic engine"
DEFAULT_DESCRIPTION =

the default description of the engine

"useful for when you need to use Anthropic AI to answer questions. " \
"You should ask targeted questions"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name: DEFAULT_NAME, description: DEFAULT_DESCRIPTION, prompts: [], **kwargs) ⇒ Anthropic

A engine is the driver for a single tool to run.

Parameters:

  • name (String) (defaults to: DEFAULT_NAME) —

    The name of the engine. Defaults to “OpenAI engine”.

  • description (String) (defaults to: DEFAULT_DESCRIPTION) —

    A description of the engine. Defaults to: useful for when you need to use AI to answer questions. You should ask targeted questions“.

  • prompts (Array<String>) (defaults to: []) —

    The prompts to use when asking the engine. Defaults to [].



28
29
30
31
32
33
 
# File 'lib/boxcars/engine/anthropic.rb', line 28

def initialize(name: DEFAULT_NAME, description: DEFAULT_DESCRIPTION, prompts: [], **kwargs)
  @llm_params = DEFAULT_PARAMS.merge(kwargs)
  @prompts = prompts
  @batch_size = 20
  super(description: description, name: name)
end

Instance Attribute Details

#batch_size ⇒ Object (readonly)

Returns the value of attribute batch_size.



8
9
10
 
# File 'lib/boxcars/engine/anthropic.rb', line 8

def batch_size
  @batch_size
end

#llm_params ⇒ Object (readonly)

Returns the value of attribute llm_params.



8
9
10
 
# File 'lib/boxcars/engine/anthropic.rb', line 8

def llm_params
  @llm_params
end

#model_kwargs ⇒ Object (readonly)

Returns the value of attribute model_kwargs.



8
9
10
 
# File 'lib/boxcars/engine/anthropic.rb', line 8

def model_kwargs
  @model_kwargs
end

#prompts ⇒ Object (readonly)

Returns the value of attribute prompts.



8
9
10
 
# File 'lib/boxcars/engine/anthropic.rb', line 8

def prompts
  @prompts
end

Instance Method Details

#anthropic_client(anthropic_api_key: nil) ⇒ Object 



39
40
41
 
# File 'lib/boxcars/engine/anthropic.rb', line 39

def anthropic_client(anthropic_api_key: nil)
  ::Anthropic::Client.new(access_token: anthropic_api_key)
end

#check_response(response, must_haves: %w[completion])) ⇒ Object

make sure we got a valid response

Parameters:

  • response (Hash) —

    The response to check.

  • must_haves (Array<String>) (defaults to: %w[completion])) —

    The keys that must be in the response. Defaults to %w.

Raises:

  • (KeyError) —

    if there is an issue with the access token.

  • (ValueError) —

    if the response is not valid.



112
113
114
115
116
117
118
119
120
121
122
123
124
 
# File 'lib/boxcars/engine/anthropic.rb', line 112

def check_response(response, must_haves: %w[completion])
  if response['error']
    code = response.dig('error', 'code')
    msg = response.dig('error', 'message') || 'unknown error'
    raise KeyError, "ANTHOPIC_API_KEY not valid" if code == 'invalid_api_key'

    raise ValueError, "Anthropic error: #{msg}"
  end

  must_haves.each do |key|
    raise ValueError, "Expecting key #{key} in response" unless response.key?(key)
  end
end

#client(prompt:, inputs: {}, **kwargs) ⇒ Object

Get an answer from the engine.

Parameters:

  • prompt (String) —

    The prompt to use when asking the engine.

  • anthropic_api_key (String) —

    Optional api key to use when asking the engine. Defaults to Boxcars.configuration.anthropic_api_key.

  • kwargs (Hash) —

    Additional parameters to pass to the engine if wanted.



48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/boxcars/engine/anthropic.rb', line 48

def client(prompt:, inputs: {}, **kwargs)
  model_params = llm_params.merge(kwargs)
  api_key = Boxcars.configuration.anthropic_api_key(**kwargs)
  aclient = anthropic_client(anthropic_api_key: api_key)
  prompt = prompt.first if prompt.is_a?(Array)

  if conversation_model?(model_params[:model])
    params = convert_to_anthropic(prompt.as_messages(inputs).merge(model_params))
    if Boxcars.configuration.log_prompts
      Boxcars.debug(params[:messages].last(2).map { |p| ">>>>>> Role: #{p[:role]} <<<<<<\n#{p[:content]}" }.join("\n"), :cyan)
    end
    response = aclient.messages(parameters: params)
    response['completion'] = response.dig('content', 0, 'text')
    response.delete('content')
    response
  else
    params = prompt.as_prompt(inputs: inputs, prefixes: default_prefixes, show_roles: true).merge(model_params)
    params[:prompt] = "\n\n#{params[:prompt]}" unless params[:prompt].start_with?("\n\n")
    params[:stop_sequences] = params.delete(:stop) if params.key?(:stop)
    Boxcars.debug("Prompt after formatting:#{params[:prompt]}", :cyan) if Boxcars.configuration.log_prompts
    aclient.complete(parameters: params)
  end
end

#combine_assistant(params) ⇒ Object 



203
204
205
206
207
 
# File 'lib/boxcars/engine/anthropic.rb', line 203

def combine_assistant(params)
  params[:messages] = combine_assistant_entries(params[:messages])
  params[:messages].last[:content].rstrip! if params[:messages].last[:role] == :assistant
  params
end

#combine_assistant_entries(hashes) ⇒ Object

if we have multiple assistant entries in a row, we need to combine them



210
211
212
213
214
215
216
217
218
219
220
 
# File 'lib/boxcars/engine/anthropic.rb', line 210

def combine_assistant_entries(hashes)
  combined_hashes = []
  hashes.each do |hash|
    if combined_hashes.empty? || combined_hashes.last[:role] != :assistant || hash[:role] != :assistant
      combined_hashes << hash
    else
      combined_hashes.last[:content].concat("\n", hash[:content].rstrip)
    end
  end
  combined_hashes
end

#conversation_model?(model) ⇒ Boolean

Returns:

  • (Boolean) 


35
36
37
 
# File 'lib/boxcars/engine/anthropic.rb', line 35

def conversation_model?(model)
  @conversation_model ||= (extract_model_version(model) > 3.49)
end

#convert_to_anthropic(params) ⇒ Object

convert generic parameters to Anthopic specific ones



196
197
198
199
200
201
 
# File 'lib/boxcars/engine/anthropic.rb', line 196

def convert_to_anthropic(params)
  params[:stop_sequences] = params.delete(:stop) if params.key?(:stop)
  params[:system] = params[:messages].shift[:content] if params.dig(:messages, 0, :role) == :system
  params[:messages].pop if params[:messages].last[:content].blank?
  combine_assistant(params)
end

#default_params ⇒ Object

Get the default parameters for the engine.



88
89
90
 
# File 'lib/boxcars/engine/anthropic.rb', line 88

def default_params
  llm_params
end

#default_prefixes ⇒ Object 



222
223
224
 
# File 'lib/boxcars/engine/anthropic.rb', line 222

def default_prefixes
  { system: "Human: ", user: "Human: ", assistant: "Assistant: ", history: :history }
end

#engine_type ⇒ Object

the engine type



156
157
158
 
# File 'lib/boxcars/engine/anthropic.rb', line 156

def engine_type
  "claude"
end

#extract_model_version(model_string) ⇒ Object

Raises:



182
183
184
185
186
187
188
189
190
191
192
193
 
# File 'lib/boxcars/engine/anthropic.rb', line 182

def extract_model_version(model_string)
  # Use a regular expression to find the version number
  match = model_string.match(/claude-(\d+)(?:-(\d+))?/)

  raise ArgumentError, "No version number found in model string: #{model_string}" unless match

  major = match[1].to_i
  minor = match[2].to_i

  # Combine major and minor versions
  major + (minor.to_f / 10)
end

#generate(prompts:, stop: nil) ⇒ EngineResult

Call out to OpenAI’s endpoint with k unique prompts.

Parameters:

  • prompts (Array<String>) —

    The prompts to pass into the model.

  • inputs (Array<String>) —

    The inputs to subsitite into the prompt.

  • stop (Array<String>) (defaults to: nil) —

    Optional list of stop words to use when generating.

Returns:



131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
 
# File 'lib/boxcars/engine/anthropic.rb', line 131

def generate(prompts:, stop: nil)
  params = {}
  params[:stop] = stop if stop
  choices = []
  # Get the token usage from the response.
  # Includes prompt, completion, and total tokens used.
  prompts.each_slice(batch_size) do |sub_prompts|
    sub_prompts.each do |sprompts, inputs|
      response = client(prompt: sprompts, inputs: inputs, **params)
      check_response(response)
      choices << response
    end
  end

  n = params.fetch(:n, 1)
  generations = []
  prompts.each_with_index do |_prompt, i|
    sub_choices = choices[i * n, (i + 1) * n]
    generations.push(generation_info(sub_choices))
  end
  EngineResult.new(generations: generations, engine_output: { token_usage: {} })
end

#generation_info(sub_choices) ⇒ Array<Generation>

Get generation informaton

Parameters:

  • sub_choices (Array<Hash>) —

    The choices to get generation info for.

Returns:

  • (Array<Generation>) —

    The generation information.



95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/boxcars/engine/anthropic.rb', line 95

def generation_info(sub_choices)
  sub_choices.map do |choice|
    Generation.new(
      text: choice["completion"],
      generation_info: {
        finish_reason: choice.fetch("stop_reason", nil),
        logprobs: choice.fetch("logprobs", nil)
      }
    )
  end
end

#get_num_tokens(text:) ⇒ Object

calculate the number of tokens used



161
162
163
 
# File 'lib/boxcars/engine/anthropic.rb', line 161

def get_num_tokens(text:)
  text.split.length # TODO: hook up to token counting gem
end

#max_tokens_for_prompt(prompt_text) ⇒ Integer

Calculate the maximum number of tokens possible to generate for a prompt.

Parameters:

  • prompt_text (String) —

    The prompt text to use.

Returns:

  • (Integer) —

    the number of tokens possible to generate.



174
175
176
177
178
179
180
 
# File 'lib/boxcars/engine/anthropic.rb', line 174

def max_tokens_for_prompt(prompt_text)
  num_tokens = get_num_tokens(prompt_text)

  # get max context size for model by name
  max_size = modelname_to_contextsize(model_name)
  max_size - num_tokens
end

#modelname_to_contextsize(_modelname) ⇒ Object

lookup the context size for a model by name

Parameters:

  • modelname (String) —

    The name of the model to lookup.



167
168
169
 
# File 'lib/boxcars/engine/anthropic.rb', line 167

def modelname_to_contextsize(_modelname)
  100000
end

#run(question, **kwargs) ⇒ Object

get an answer from the engine for a question.

Parameters:

  • question (String) —

    The question to ask the engine.

  • kwargs (Hash) —

    Additional parameters to pass to the engine if wanted.

Raises:



75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/boxcars/engine/anthropic.rb', line 75

def run(question, **kwargs)
  prompt = Prompt.new(template: question)
  response = client(prompt: prompt, **kwargs)

  raise Error, "Anthropic: No response from API" unless response
  raise Error, "Anthropic: #{response['error']}" if response['error']

  answer = response['completion']
  Boxcars.debug(response, :yellow)
  answer
end