Aider Polyglot Leaderboard

What this benchmark measures

• multi-language code editing
• instruction following in code
• edit-format compliance

Relevant to:

• coding assistant selection
• autonomous code-edit screening

Failure modes it exercises:

• failed edits and malformed edit output across languages

Method and its limits

225 Exercism coding exercises across six languages, run through the aider harness, reporting percent correct along with edit-format compliance and cost per run. Aider's benchmark documentation defines each pass-rate statistic as the percent of tasks for which all tests passed, with one such statistic per value of the --tries parameter, so pass rate 1 covers the first attempt and pass rate 2 allows a second.

• Results are harness-specific: the aider scaffolding, edit format, and retry behaviour are part of what is measured.
• The published leaderboard reports a headline percentage alongside two pass rates without fully defining them on the page, so the exact retry semantics behind a score are not established from the source.
• Exercises are self-contained, so repository-scale context handling is not tested.

Dataset

Dataset

aider polyglot benchmark

Type

Exercism coding exercises across six languages

Freshness

current

Benchmark release

2026-10-11

225 exercises across C++, Go, Java, JavaScript, Python, and Rust as registered on the official leaderboard. The leaderboard is continuously updated, so the figure reflects the date accessed rather than a fixed release.

How to read this score

Percent correct is interpretable only together with edit-format compliance and the harness version; the leaderboard also reports cost, which materially differentiates models at similar accuracy.

Similarity to real tasks: Medium — Established from the primary source: the benchmark runs 225 Exercism exercises across C++, Go, Java, JavaScript, Python, and Rust, and measures whether a model can follow instructions and edit code without human intervention. Editing existing files across six languages resembles assistant work more closely than function-synthesis benchmarks do. Exercism exercises are self-contained problems, not repository-scale changes.

Data contamination risk: High — Exercism exercises and their published solutions are public and long-standing, so training overlap is likely. No run-specific overlap audit is registered.

Benchmark gaming risk: Medium — Because the harness is part of the measurement, scaffolding and edit-format tuning change results without changing underlying coding ability.

What you still need to test yourself

• Evaluate on your own repository, where context size and build systems dominate.
• Re-test with your own tooling; results are specific to the aider harness.

This benchmark supports decisions about:

• Compare coding assistants on editing rather than greenfield synthesis.
• Weigh accuracy against the cost per run that the leaderboard reports.

Limitations

• Harness-specific; not a property of the model alone.
• Not evidence of repository-scale engineering ability.

Official leaderboard accessed 2026-10-11. The leaderboard is continuously updated, so scores recorded here can drift from the published table.

What this evidence does not cover

• Two models are currently scored on this page, and neither the leaderboard revision nor the harness version behind those scores is registered.

Sources

• Aider LLM Leaderboards Aider · accessed 2026-10-11Supports the 225-exercise, six-language Exercism composition and the stated purpose of measuring instruction following and code editing without human intervention. This is a continuously updated leaderboard, so the record describes the published revision as accessed on 2026-10-11 rather than a fixed dataset release; it does not establish the retry semantics behind the reported pass rates, and does not support any score recorded on this page.
• aider benchmark methodology and pass-rate definition Aider · accessed 2026-10-25Aider's own benchmark documentation, which defines the pass-rate statistics and the retry parameter that produces more than one of them. It establishes what each reported pass rate means: the percent of tasks for which all tests passed, one statistic per value of the --tries parameter, so pass rate 1 covers the first attempt and pass rate 2 allows a second. It does not record which leaderboard revision or harness version produced the scores shown here.

How this benchmark is scored

Primary source: https://aider.chat/docs/leaderboards/

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Aider Polyglot Leaderboard.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path