Holistic Evaluation of Text-To-Image Models
by Stanford CRFM · open-source · Last verified 2026-03-17
Presents HEIM (Holistic Evaluation of Image Models), a comprehensive evaluation framework for text-to-image models assessing 12 aspects including alignment, quality, aesthetics, originality, reasoning, knowledge, bias, toxicity, and fairness. Evaluates 26 models, revealing that no single model excels across all aspects and exposing significant safety gaps.
https://arxiv.org/abs/2311.04287 ↗B
B—Above Average
Adoption: BQuality: AFreshness: BCitations: BEngagement: F
Specifications
- License
- Apache-2.0
- Pricing
- open-source
- Capabilities
- text-to-image-evaluation, multi-aspect-assessment, bias-evaluation, safety-evaluation, quality-measurement
- Integrations
- Use Cases
- model-evaluation, responsible-ai, safety-research, research
- API Available
- No
- Tags
- evaluation, text-to-image, holistic, benchmark, multimodal, safety
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
61.8Adoption
68
Quality
88
Freshness
68
Citations
68
Engagement
0
Put AI to work for your business
Deploy this paper alongside autonomous AaaS agents that handle tasks end-to-end — no babysitting required.