Class: Langchain::Evals::Ragas::ContextRelevance

Inherits:
Object
  • Object
show all
Defined in:
lib/langchain/evals/ragas/context_relevance.rb

Overview

Context Relevance refers to the idea that the retrieved context should be focused, containing as little irrelevant information as possible.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(llm:) ⇒ ContextRelevance

Returns a new instance of ContextRelevance.

Parameters:



13
14
15
 
# File 'lib/langchain/evals/ragas/context_relevance.rb', line 13

def initialize(llm:)
  @llm = llm
end

Instance Attribute Details

#llm ⇒ Object (readonly)

Returns the value of attribute llm.



10
11
12
 
# File 'lib/langchain/evals/ragas/context_relevance.rb', line 10

def llm
  @llm
end

Instance Method Details

#score(question:, context:) ⇒ Float

Returns Context Relevance score.

Parameters:

  • question (String) —

    Question

  • context (String) —

    Context

Returns:

  • (Float) —

    Context Relevance score



20
21
22
23
24
25
26
27
28
 
# File 'lib/langchain/evals/ragas/context_relevance.rb', line 20

def score(question:, context:)
  prompt = context_relevance_prompt_template.format(
    question: question,
    context: context
  )
  sentences = llm.complete(prompt: prompt).completion

  (sentence_count(sentences).to_f / sentence_count(context).to_f)
end