HellaSwag Leaderboard

What this benchmark measures

• commonsense natural language inference
• plausible continuation selection

Relevant to:

• base-model capability screening
• open-model comparison

Failure modes it exercises:

• commonsense errors in short continuation tasks

Method and its limits

Four-way multiple-choice continuation selection, with incorrect options machine-generated and iteratively filtered by discriminators to be hard for models while remaining obvious to humans.

• Adversarial Filtering targets the models of 2019, so the difficulty profile is tuned to an older generation.
• The benchmark is effectively saturated: the paper recorded humans above 95% against models below 48%, and that gap has since closed.

Dataset

Dataset

HellaSwag

Type

adversarially filtered multiple-choice continuations

Freshness

aging

Benchmark release

2019-05-19

The primary paper registers the Adversarial Filtering construction and the human and model accuracy figures of the time. The date records paper submission.

How to read this score

Accuracy is interpretable only as a floor check. Because frontier models now score near the human ceiling reported in the paper, differences at the top carry little information.

Similarity to real tasks: Low — Established from the primary methodology: HellaSwag asks a model to choose the most likely continuation of an event description from machine-generated candidates selected by Adversarial Filtering. It is a controlled multiple-choice probe of commonsense plausibility and does not resemble any production task; no generation, tool use, retrieval, or multi-turn behaviour is exercised.

Data contamination risk: High — HellaSwag has been public since 2019 and is a standard component of open-model evaluation suites, so training overlap is likely. No run-specific audit is registered.

Benchmark gaming risk: High — The benchmark appears on open-model leaderboards that are directly optimised against, and scoring is sensitive to normalisation choices for multiple-choice likelihoods.

What you still need to test yourself

• Evaluate on your actual task; HellaSwag ranking does not transfer to instruction-following or generation quality.

This benchmark supports decisions about:

• Use as a floor check on an open base model.
• Detect gross commonsense failure, not relative quality.

Limitations

• Saturated: it no longer separates capable models.
• Not evidence of generation quality, instruction following, or reasoning.

Primary source reviewed 2026-10-11. HellaSwag is treated as an aging and effectively saturated 2019 benchmark.

What this evidence does not cover

• Only one model is currently scored on this page, so no comparison is supported.

Sources

• HellaSwag: Can a Machine Really Finish Your Sentence? Zellers, Holtzman, Bisk, Farhadi, Choi · accessed 2026-10-11Supports the Adversarial Filtering construction and the stated human (>95%) and contemporary model (<48%) accuracy figures; it does not support any model score shown here.
• HellaSwag project page and test-set leaderboard Zellers et al. (HellaSwag authors) · accessed 2026-10-25The authors' project page, carrying the paper, the dataset links and the test-set leaderboard, which records that submissions closed in November 2024. It establishes the benchmark's official reporting surface; it does not state the dataset size, and the authors' GitHub repository is currently blocked on GitHub under a DMCA notice, so the project page is the citable official surface.

How this benchmark is scored

Primary source: https://rowanzellers.com/hellaswag/

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to HellaSwag Leaderboard.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path