BioASQ
by Tsatsaronis et al. / BioASQ Challenge · open-source · Last verified 2026-03-17
BioASQ is a large-scale biomedical semantic question-answering benchmark that combines document retrieval, concept mapping, and answer extraction from PubMed literature. It challenges models across yes/no, factoid, list, and summary answer types, with gold-standard answers curated by biomedical experts.
http://bioasq.org ↗B
B—Above Average
Adoption: B+Quality: AFreshness: ACitations: AEngagement: F
Specifications
- License
- CC BY 2.5
- Pricing
- open-source
- Capabilities
- evaluation, biomedical-qa, information-retrieval, scientific-reasoning
- Integrations
- Use Cases
- model-evaluation, biomedical-nlp, clinical-decision-support
- API Available
- No
- Evaluated Models
- gpt-4o, claude-opus-4, biogpt, llama-3-70b
- Metrics
- exact-match, mean-average-precision, f-measure
- Methodology
- Annual challenge with Phase A (document/snippet retrieval) and Phase B (answer generation). Expert biomedical curators create questions; gold snippets and answers used for evaluation. Factoid EM, list F-measure, and yes/no accuracy reported per type.
- Last Run
- 2026-03-05
- Tags
- biomedical, qa, information-retrieval, pubmed, expert
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
67.7Adoption
74
Quality
88
Freshness
80
Citations
82
Engagement
0