ModelRefs / SWE-Bench Lite Methodology and Evidence
SWE-Bench Lite Methodology and Evidence
300-task subset of SWE-Bench for faster eval. Methodology, dataset provenance, and citation sources for SWE-Bench Lite.
Overview
300-task subset of SWE-Bench for faster eval.
How it is measured: Resolve-rate using agentless or agent pipelines.
How this benchmark is scored
| Category | coding |
|---|---|
| Maximum score | 100 % resolved |
| Direction | Higher is better |
Primary source: https://www.swebench.com/lite.html
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to SWE-Bench Lite Methodology and Evidence.
Frequently asked questions
What does the SWE-Bench Lite benchmark measure?
300-task subset of SWE-Bench for faster eval.
What is the methodology?
Resolve-rate using agentless or agent pipelines.