Class: Boxcars::Engine Abstract

Inherits:
Object
  • Object
show all
Defined in:
lib/boxcars/engine.rb

Overview

This class is abstract.

Direct Known Subclasses

Anthropic, Cohere, Gpt4allEng, Groq, Ollama, Openai, Perplexityai

Instance Method Summary collapse

Constructor Details

#initialize(description: 'Engine', name: nil) ⇒ Engine

An Engine is used by Boxcars to generate output from prompts

Parameters:

  • name (String) (defaults to: nil) —

    The name of the Engine. Defaults to classname.

  • description (String) (defaults to: 'Engine') —

    A description of the Engine.



9
10
11
12
 
# File 'lib/boxcars/engine.rb', line 9

def initialize(description: 'Engine', name: nil)
  @name = name || self.class.name
  @description = description
end

Instance Method Details

#generate(prompts:, stop: nil) ⇒ EngineResult

Call out to OpenAI’s endpoint with k unique prompts.

Parameters:

  • prompts (Array<String>) —

    The prompts to pass into the model.

  • inputs (Array<String>) —

    The inputs to subsitite into the prompt.

  • stop (Array<String>) (defaults to: nil) —

    Optional list of stop words to use when generating.

Returns:



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/boxcars/engine.rb', line 45

def generate(prompts:, stop: nil)
  params = {}
  params[:stop] = stop if stop
  choices = []
  token_usage = {}
  # Get the token usage from the response.
  # Includes prompt, completion, and total tokens used.
  inkeys = %w[completion_tokens prompt_tokens total_tokens].freeze
  prompts.each_slice(batch_size) do |sub_prompts|
    sub_prompts.each do |sprompts, inputs|
      response = client(prompt: sprompts, inputs: inputs, **params)
      check_response(response)
      choices.concat(response["choices"])
      usage_keys = inkeys & response["usage"].keys
      usage_keys.each { |key| token_usage[key] = token_usage[key].to_i + response["usage"][key] }
    end
  end

  n = params.fetch(:n, 1)
  generations = []
  prompts.each_with_index do |_prompt, i|
    sub_choices = choices[i * n, (i + 1) * n]
    generations.push(generation_info(sub_choices))
  end
  EngineResult.new(generations: generations, engine_output: { token_usage: token_usage })
end

#generation_info(sub_choices) ⇒ Array<Generation>

Get generation informaton

Parameters:

  • sub_choices (Array<Hash>) —

    The choices to get generation info for.

Returns:

  • (Array<Generation>) —

    The generation information.



28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/boxcars/engine.rb', line 28

def generation_info(sub_choices)
  sub_choices.map do |choice|
    Generation.new(
      text: choice.dig("message", "content") || choice["text"],
      generation_info: {
        finish_reason: choice.fetch("finish_reason", nil),
        logprobs: choice.fetch("logprobs", nil)
      }
    )
  end
end

#get_num_tokens(text:) ⇒ Object

calculate the number of tokens used



21
22
23
 
# File 'lib/boxcars/engine.rb', line 21

def get_num_tokens(text:)
  text.split.length # TODO: hook up to token counting gem
end

#run(question) ⇒ Object

Get an answer from the Engine.

Parameters:

  • question (String) —

    The question to ask the Engine.

Raises:

  • (NotImplementedError) 


16
17
18
 
# File 'lib/boxcars/engine.rb', line 16

def run(question)
  raise NotImplementedError
end