Class: Langchain::LLM::LlamaCpp
- Defined in:
- lib/langchain/llm/llama_cpp.rb
Overview
Instance Attribute Summary collapse
-
#model_path ⇒ Object
Returns the value of attribute model_path.
-
#n_ctx ⇒ Object
Returns the value of attribute n_ctx.
-
#n_gpu_layers ⇒ Object
Returns the value of attribute n_gpu_layers.
-
#n_threads ⇒ Object
writeonly
Sets the attribute n_threads.
-
#seed ⇒ Object
Returns the value of attribute seed.
Attributes inherited from Base
Instance Method Summary collapse
-
#complete(prompt:, n_predict: 128) ⇒ String
The completed prompt.
-
#embed(text:) ⇒ Array<Float>
The embedding.
-
#initialize(model_path:, n_gpu_layers: 1, n_ctx: 2048, n_threads: 1, seed: 0) ⇒ LlamaCpp
constructor
A new instance of LlamaCpp.
Methods inherited from Base
#chat, #chat_parameters, #default_dimension, #default_dimensions, #summarize
Methods included from DependencyHelper
Constructor Details
#initialize(model_path:, n_gpu_layers: 1, n_ctx: 2048, n_threads: 1, seed: 0) ⇒ LlamaCpp
Returns a new instance of LlamaCpp.
25 26 27 28 29 30 31 32 33 |
# File 'lib/langchain/llm/llama_cpp.rb', line 25 def initialize(model_path:, n_gpu_layers: 1, n_ctx: 2048, n_threads: 1, seed: 0) depends_on "llama_cpp" @model_path = model_path @n_gpu_layers = n_gpu_layers @n_ctx = n_ctx @n_threads = n_threads @seed = seed end |
Instance Attribute Details
#model_path ⇒ Object
Returns the value of attribute model_path.
17 18 19 |
# File 'lib/langchain/llm/llama_cpp.rb', line 17 def model_path @model_path end |
#n_ctx ⇒ Object
Returns the value of attribute n_ctx.
17 18 19 |
# File 'lib/langchain/llm/llama_cpp.rb', line 17 def n_ctx @n_ctx end |
#n_gpu_layers ⇒ Object
Returns the value of attribute n_gpu_layers.
17 18 19 |
# File 'lib/langchain/llm/llama_cpp.rb', line 17 def n_gpu_layers @n_gpu_layers end |
#n_threads=(value) ⇒ Object
Sets the attribute n_threads
18 19 20 |
# File 'lib/langchain/llm/llama_cpp.rb', line 18 def n_threads=(value) @n_threads = value end |
#seed ⇒ Object
Returns the value of attribute seed.
17 18 19 |
# File 'lib/langchain/llm/llama_cpp.rb', line 17 def seed @seed end |
Instance Method Details
#complete(prompt:, n_predict: 128) ⇒ String
Returns The completed prompt.
51 52 53 54 55 |
# File 'lib/langchain/llm/llama_cpp.rb', line 51 def complete(prompt:, n_predict: 128) # contexts do not appear to be stateful when it comes to completion, so re-use the same one context = completion_context ::LLaMACpp.generate(context, prompt, n_predict: n_predict) end |
#embed(text:) ⇒ Array<Float>
Returns The embedding.
37 38 39 40 41 42 43 44 45 46 |
# File 'lib/langchain/llm/llama_cpp.rb', line 37 def (text:) # contexts are kinda stateful when it comes to embeddings, so allocate one each time context = = @model.tokenize(text: text, add_bos: true) return unless .size.positive? context.eval(tokens: , n_past: 0) Langchain::LLM::LlamaCppResponse.new(context, model: context.model.desc) end |