Class: Boxcars::Cohere

Inherits:
Engine
  • Object
show all
Defined in:
lib/boxcars/engine/cohere.rb

Overview

A engine that uses Cohere’s API.

Constant Summary collapse

DEFAULT_PARAMS =

The default parameters to use when asking the engine.

{
  model: "command-r-plus",
  max_tokens: 4000,
  max_input_tokens: 1000,
  temperature: 0.2
}.freeze
DEFAULT_NAME =

the default name of the engine

"Cohere engine"
DEFAULT_DESCRIPTION =

the default description of the engine

"useful for when you need to use Cohere AI to answer questions. " \
"You should ask targeted questions"

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from Engine

#generate, #generation_info, #get_num_tokens

Constructor Details

#initialize(name: DEFAULT_NAME, description: DEFAULT_DESCRIPTION, prompts: [], **kwargs) ⇒ Cohere

A engine is the driver for a single tool to run.

Parameters:

  • name (String) (defaults to: DEFAULT_NAME) —

    The name of the engine. Defaults to “OpenAI engine”.

  • description (String) (defaults to: DEFAULT_DESCRIPTION) —

    A description of the engine. Defaults to: useful for when you need to use AI to answer questions. You should ask targeted questions“.

  • prompts (Array<String>) (defaults to: []) —

    The prompts to use when asking the engine. Defaults to [].



28
29
30
31
32
33
 
# File 'lib/boxcars/engine/cohere.rb', line 28

def initialize(name: DEFAULT_NAME, description: DEFAULT_DESCRIPTION, prompts: [], **kwargs)
  @llm_params = DEFAULT_PARAMS.merge(kwargs)
  @prompts = prompts
  @batch_size = 20
  super(description: description, name: name)
end

Instance Attribute Details

#batch_size ⇒ Object (readonly)

Returns the value of attribute batch_size.



7
8
9
 
# File 'lib/boxcars/engine/cohere.rb', line 7

def batch_size
  @batch_size
end

#llm_params ⇒ Object (readonly)

Returns the value of attribute llm_params.



7
8
9
 
# File 'lib/boxcars/engine/cohere.rb', line 7

def llm_params
  @llm_params
end

#model_kwargs ⇒ Object (readonly)

Returns the value of attribute model_kwargs.



7
8
9
 
# File 'lib/boxcars/engine/cohere.rb', line 7

def model_kwargs
  @model_kwargs
end

#prompts ⇒ Object (readonly)

Returns the value of attribute prompts.



7
8
9
 
# File 'lib/boxcars/engine/cohere.rb', line 7

def prompts
  @prompts
end

Instance Method Details

#chat(params, cohere_api_key) ⇒ Object 



39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'lib/boxcars/engine/cohere.rb', line 39

def chat(params, cohere_api_key)
  raise Boxcars::ConfigurationError('Cohere API key not set') if cohere_api_key.blank?

  # Define the API endpoint and parameters
  api_endpoint = 'https://api.cohere.ai/v1/chat'

  connection = Faraday.new(api_endpoint) do |faraday|
    faraday.request :url_encoded
    faraday.headers['Authorization'] = "Bearer #{cohere_api_key}"
    faraday.headers['Content-Type'] = 'application/json'
  end

  # Make the API call
  response = connection.post { |req| req.body = params.to_json }

  # response_data = JSON.parse(response.body, symbolize_names: true)
  # response_data[:text]
  JSON.parse(response.body, symbolize_names: true)
end

#check_response(response, must_haves: %w[completion])) ⇒ Object

make sure we got a valid response

Parameters:

  • response (Hash) —

    The response to check.

  • must_haves (Array<String>) (defaults to: %w[completion])) —

    The keys that must be in the response. Defaults to %w.

Raises:

  • (KeyError) —

    if there is an issue with the access token.

  • (ValueError) —

    if the response is not valid.



98
99
100
101
102
103
104
105
106
107
108
109
110
 
# File 'lib/boxcars/engine/cohere.rb', line 98

def check_response(response, must_haves: %w[completion])
  if response['error']
    code = response.dig('error', 'code')
    msg = response.dig('error', 'message') || 'unknown error'
    raise KeyError, "ANTHOPIC_API_KEY not valid" if code == 'invalid_api_key'

    raise ValueError, "Cohere error: #{msg}"
  end

  must_haves.each do |key|
    raise ValueError, "Expecting key #{key} in response" unless response.key?(key)
  end
end

#client(prompt:, inputs: {}, **kwargs) ⇒ Object

Get an answer from the engine.

Parameters:

  • prompt (String) —

    The prompt to use when asking the engine.

  • cohere_api_key (String) —

    Optional api key to use when asking the engine. Defaults to Boxcars.configuration.cohere_api_key.

  • kwargs (Hash) —

    Additional parameters to pass to the engine if wanted.



64
65
66
67
68
69
70
71
 
# File 'lib/boxcars/engine/cohere.rb', line 64

def client(prompt:, inputs: {}, **kwargs)
  api_key = Boxcars.configuration.cohere_api_key(**kwargs)
  params = prompt.as_prompt(inputs: inputs, prefixes: default_prefixes, show_roles: true).merge(llm_params.merge(kwargs))
  params[:message] = params.delete(:prompt)
  params[:stop_sequences] = params.delete(:stop) if params.key?(:stop)
  Boxcars.debug("Prompt after formatting:#{params[:message]}", :cyan) if Boxcars.configuration.log_prompts
  chat(params, api_key)
end

#conversation_model?(_model) ⇒ Boolean

Returns:

  • (Boolean) 


35
36
37
 
# File 'lib/boxcars/engine/cohere.rb', line 35

def conversation_model?(_model)
  true
end

#default_params ⇒ Object

Get the default parameters for the engine.



89
90
91
 
# File 'lib/boxcars/engine/cohere.rb', line 89

def default_params
  llm_params
end

#default_prefixes ⇒ Object 



134
135
136
 
# File 'lib/boxcars/engine/cohere.rb', line 134

def default_prefixes
  { system: "SYSTEM: ", user: "USER: ", assistant: "CHATBOT: ", history: :history }
end

#engine_type ⇒ Object

the engine type



113
114
115
 
# File 'lib/boxcars/engine/cohere.rb', line 113

def engine_type
  "claude"
end

#max_tokens_for_prompt(prompt_text) ⇒ Integer

Calculate the maximum number of tokens possible to generate for a prompt.

Parameters:

  • prompt_text (String) —

    The prompt text to use.

Returns:

  • (Integer) —

    the number of tokens possible to generate.



126
127
128
129
130
131
132
 
# File 'lib/boxcars/engine/cohere.rb', line 126

def max_tokens_for_prompt(prompt_text)
  num_tokens = get_num_tokens(prompt_text)

  # get max context size for model by name
  max_size = modelname_to_contextsize(model_name)
  max_size - num_tokens
end

#modelname_to_contextsize(_modelname) ⇒ Object

lookup the context size for a model by name

Parameters:

  • modelname (String) —

    The name of the model to lookup.



119
120
121
 
# File 'lib/boxcars/engine/cohere.rb', line 119

def modelname_to_contextsize(_modelname)
  100000
end

#run(question, **kwargs) ⇒ Object

get an answer from the engine for a question.

Parameters:

  • question (String) —

    The question to ask the engine.

  • kwargs (Hash) —

    Additional parameters to pass to the engine if wanted.

Raises:



76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/boxcars/engine/cohere.rb', line 76

def run(question, **kwargs)
  prompt = Prompt.new(template: question)
  response = client(prompt: prompt, **kwargs)

  raise Error, "Cohere: No response from API" unless response
  raise Error, "Cohere: #{response[:error]}" if response[:error]

  answer = response[:text]
  Boxcars.debug(response, :yellow)
  answer
end