WinoGrande Leaderboard

What this benchmark measures

• commonsense coreference resolution
• pronoun disambiguation

Relevant to:

• base-model capability screening
• open-model comparison

Failure modes it exercises:

• referential ambiguity errors

Method and its limits

44k crowdsourced fill-in-the-blank coreference problems, filtered by the AfLite algorithm to remove dataset-specific word association artifacts.

• AfLite removes machine-detectable associations but cannot guarantee the remaining items are free of other shortcuts.
• The paper argues that high scores on related Winograd-style benchmarks may overstate machine commonsense ability.

Dataset

Dataset

WinoGrande

Type

debiased commonsense coreference problems

Freshness

aging

Benchmark release

2019-07-24

44k problems as registered in the primary paper, scaled up from the 273 expert-written problems of the original Winograd Schema Challenge. The date records paper submission.

How to read this score

The paper reports human performance at 94.0% against contemporary systems at 59.4-79.1% depending on training data allowed. Current frontier models score far above that range, so the benchmark now functions as a floor check.

Similarity to real tasks: Low — Established from the primary methodology: WinoGrande presents 44k fill-in-the-blank problems requiring commonsense pronoun resolution, debiased with the AfLite algorithm. It is a controlled linguistic probe rather than a production task, and exercises no generation, retrieval, or tool use.

Data contamination risk: High — Public since 2019 and standard in open-model evaluation suites, so training overlap is likely. No run-specific audit is registered.

Benchmark gaming risk: High — The benchmark is a leaderboard component that is directly optimised against, and the paper itself cautions that strong scores may overstate commonsense ability.

What you still need to test yourself

• Evaluate coreference behaviour on your own documents, which are longer and less controlled than these items.

This benchmark supports decisions about:

• Use as a floor check on an open base model.

Limitations

• Effectively saturated for frontier models.
• The authors explicitly warn that high scores may overstate commonsense ability.

Primary source reviewed 2026-10-11. WinoGrande is treated as an aging 2019 benchmark.

What this evidence does not cover

• Only one model is currently scored on this page, so no comparison is supported.

Sources

• WinoGrande: An Adversarial Winograd Schema Challenge at Scale Sakaguchi, Le Bras, Bhagavatula, Choi · accessed 2026-10-11Supports the 44k problem count, the AfLite debiasing method, the 94.0% human figure and the 59.4-79.1% contemporary model range, and the authors' caution about overstated commonsense ability; it does not support any model score shown here.
• WinoGrande dataset and evaluation release Allen Institute for AI · accessed 2026-10-25AI2's official WinoGrande release, matching the paper's title. It establishes the dataset and its evaluation assets; it does not establish any model's reported accuracy.

How this benchmark is scored

Primary source: https://winogrande.allenai.org/

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to WinoGrande Leaderboard.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path