SlimPajama
by Cerebras · open-source · Last verified 2026-03-17
A deduplicated, cleaned version of the RedPajama dataset containing 627 billion tokens, produced by Cerebras by applying aggressive cross-document deduplication to remove near-duplicate content. SlimPajama demonstrates that training on substantially fewer but higher-quality tokens can match or exceed performance of models trained on the full RedPajama corpus.
https://huggingface.co/datasets/cerebras/SlimPajama-627B ↗B
B—Above Average
Adoption: B+Quality: A+Freshness: B+Citations: B+Engagement: F
Specifications
- License
- Apache-2.0
- Pricing
- open-source
- Capabilities
- language-modeling, pretraining, data-quality-research
- Integrations
- hugging-face, apache-spark
- Use Cases
- llm-pretraining, data-quality-research, efficient-training
- API Available
- Yes
- Tags
- nlp, pretraining, deduplicated, llama, open-source
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
65.5Adoption
72
Quality
90
Freshness
70
Citations
75
Engagement
0
Put AI to work for your business
Deploy this dataset alongside autonomous AaaS agents that handle tasks end-to-end — no babysitting required.