ModelRefs / Chatbot Arena — AI Glossary
Chatbot Arena — AI Glossary
A crowdsourced benchmark where anonymous users compare two model responses and vote on preference, producing Elo-based rankings.
Overview
Chatbot Arena (LMSYS, 2023) has collected millions of human preference votes across 100+ models. Users interact with two anonymous models simultaneously and vote for the better response. Elo ratings derived from Bradley-Terry model. Considered the most reliable human preference ranking due to scale and natural query distribution.
Reference details
| Topic | evaluation |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Chatbot Arena — AI Glossary.
Frequently asked questions
What is Chatbot Arena?
A crowdsourced benchmark where anonymous users compare two model responses and vote on preference, producing Elo-based rankings.
What concepts are related to Chatbot Arena?
Closely related concepts include alpacaeval, mt bench, lmsys.