ModelRefs / Regression Testing (LLM) — AI Glossary
Regression Testing (LLM) — AI Glossary
Automated tests that detect quality regressions when prompts, models, or retrieval sources change. Catches silent quality drops before production deployment.
Overview
LLM regression tests run a fixed golden dataset through a pipeline and compare outputs against previous scores using LLM-as-judge or exact-match metrics. Catches silent quality drops before production deployment.
Reference details
| Topic | evaluation |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
A regression test asks whether behaviour changed against your own previous output; an evaluation asks how good it is in absolute terms. Both are needed and they answer different questions. The complication unique to this setting is nondeterminism: identical inputs can produce different outputs, so a regression test needs a tolerance or a semantic comparison rather than the exact-match assertion that works for code.
Referenced by
This term is used by the following ModelRefs references:
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Regression Testing (LLM) — AI Glossary.
Frequently asked questions
What is Regression Testing (LLM)?
Automated tests that detect quality regressions when prompts, models, or retrieval sources change.
What concepts are related to Regression Testing (LLM)?
Closely related concepts include eval, llm as judge, llmops.