ModelRefs / RAGAS — AI Glossary
RAGAS — AI Glossary
A framework for automated evaluation of RAG pipelines using LLM judges for faithfulness, answer relevance, context precision, and recall.
Overview
RAGAS (Es et al. 2023) provides reference-free RAG evaluation metrics computed by an LLM judge. Core metrics: Faithfulness, Answer Relevance, Context Precision, Context Recall. Supports OpenAI and HuggingFace models as evaluators. Integrates with LangSmith, LlamaIndex, and Arize Phoenix for production RAG monitoring.
Reference details
| Topic | evaluation |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Example: Reference-free, and what that costs
Faithfulness asks whether each claim in the answer is supported by the retrieved context — checkable without a gold answer, which is why it can run on live traffic. Context recall cannot: it needs ground truth. So RAGAS splits naturally into metrics you can monitor continuously and metrics you can only run against a labelled set.
Commonly confused with
RAGAS scores are produced by an LLM judge, not measured. They inherit the judge's biases and its failure modes, and they move when the judge model changes. Treat them as a consistent instrument for tracking direction, not as ground truth about quality.
When to use it
Reach for it when:
- You need automated RAG evaluation in CI or on sampled live traffic
- Separating retrieval failures from generation failures systematically
- You have validated the judge against human labels at least once
Reach for something else when:
- As the sole signal — an unvalidated judge can be confidently wrong at scale
- Comparing scores across different judge models or versions
- Where the domain is specialised and the judge lacks the expertise to grade it
Primary source
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to RAGAS — AI Glossary.
Frequently asked questions
What is RAGAS?
A framework for automated evaluation of RAG pipelines using LLM judges for faithfulness, answer relevance, context precision, and recall.
What concepts are related to RAGAS?
Closely related concepts include faithfulness, answer relevance, context recall.