MMMU
by CUHK / Waterloo · open-source · Last verified 2026-03-01
Massive Multi-discipline Multimodal Understanding benchmark with 11,500 college-level problems requiring both image understanding and domain-specific reasoning across art, business, science, health, engineering, and humanities disciplines.
https://mmmu-benchmark.github.io ↗B
B—Above Average
Adoption: B+Quality: A+Freshness: ACitations: B+Engagement: F
Specifications
- License
- Apache-2.0
- Pricing
- open-source
- Capabilities
- model-evaluation, multimodal-testing, visual-reasoning-assessment
- Integrations
- lm-eval-harness
- Use Cases
- multimodal-model-comparison, visual-reasoning-evaluation, research
- API Available
- No
- Evaluated Models
- claude-4, gpt-5, gemini-2.5-pro
- Metrics
- accuracy, per-discipline-accuracy
- Methodology
- College-level multiple-choice and open-ended questions with image inputs across 30 subjects. Tests both visual understanding and domain knowledge.
- Last Run
- 2026-03-01
- Tags
- benchmark, evaluation, multimodal, reasoning, expert
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
66.9Adoption
76
Quality
90
Freshness
88
Citations
74
Engagement
0