REALM: Retrieval-Augmented Language Model Pre-Training
by Google Research · free · Last verified 2026-03-17
Proposes REALM, which augments language model pre-training with a learned textual knowledge retriever, enabling the model to retrieve and attend over documents from a large corpus during both pre-training and fine-tuning. REALM achieves state-of-the-art on Open-domain QA benchmarks while providing interpretable knowledge retrieval.
https://arxiv.org/abs/2002.08909 ↗B+
B+—Good
Adoption: B+Quality: AFreshness: C+Citations: A+Engagement: F
Specifications
- License
- Open Access
- Pricing
- free
- Capabilities
- retrieval-augmented-pretraining, open-domain-qa, interpretable-retrieval, knowledge-grounding
- Integrations
- Use Cases
- question-answering, knowledge-intensive-nlp, research
- API Available
- No
- Tags
- rag, pretraining, retrieval, open-domain-qa, knowledge
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
70.5Adoption
76
Quality
88
Freshness
55
Citations
90
Engagement
0
Put AI to work for your business
Deploy this paper alongside autonomous AaaS agents that handle tasks end-to-end — no babysitting required.