Chatbot Arena
by LMSys · free · Last verified 2026-04-24
Chatbot Arena is a crowdsourced human evaluation platform from LMSys where users anonymously compare responses from two random LLMs and vote for the better one. The resulting Elo-based leaderboard (LMSYS Leaderboard) is widely regarded as the most reliable measure of real-world LLM preference across diverse user tasks.
https://chat.lmsys.org ↗C
C—Below Average
Adoption: C+Quality: B+Freshness: ACitations: CEngagement: F
Specifications
- License
- Proprietary
- Pricing
- free
- Capabilities
- Integrations
- Use Cases
- API Available
- No
- Evaluated Models
- claude-4, gpt-5, gemini-2.5-pro, deepseek-v3, llama-4-405b
- Metrics
- elo-rating, win-rate, confidence-interval
- Methodology
- Blind A/B testing with crowdsourced human judges. Users chat with two anonymous models and vote for the preferred response. Elo ratings computed from pairwise comparisons.
- Last Run
- 2026-03-15
- Tags
- benchmark, human-evaluation, elo, leaderboard, preference, crowdsourced, lmsys
- Added
- 2026-04-24
- Completeness
- 60%
Index Score
44Adoption
50
Quality
70
Freshness
80
Citations
40
Engagement
0