ModelRefs / Benchmark Contamination — AI Glossary
Benchmark Contamination — AI Glossary
The inflation of benchmark scores when evaluation examples appear in model training data, invalidating the measurement.
Overview
Contamination occurs when benchmark test sets (MMLU, HumanEval, GSM8K) are present in Common Crawl or GitHub-scraped training data. Detection: n-gram overlap analysis between training corpus and benchmark. Mitigation: contamination-resistant benchmarks (LiveCodeBench uses post-cutoff problems), held-out test sets, and contamination reporting in model cards.
Reference details
| Topic | ecosystem |
|---|---|
| Also known as | data contamination, test set leakage |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Benchmark Contamination — AI Glossary.
Frequently asked questions
What is Benchmark Contamination?
The inflation of benchmark scores when evaluation examples appear in model training data, invalidating the measurement.
Is Benchmark Contamination the same as data contamination?
Yes — data contamination, test set leakage are common aliases for Benchmark Contamination.
What concepts are related to Benchmark Contamination?
Closely related concepts include data contamination, evaluation harness, data deduplication.