Generating Long Sequences with Sparse Transformers
by OpenAI · free · Last verified 2026-03-17
Introduces Sparse Transformers, which use factored sparse attention patterns to reduce attention complexity from O(n²) to O(n√n), enabling transformers to model sequences thousands of steps long. Applied to autoregressive density estimation of images, text, and audio.
https://arxiv.org/abs/1904.10509 ↗B
B—Above Average
Adoption: BQuality: AFreshness: DCitations: BEngagement: F
Specifications
- License
- Open Access
- Pricing
- free
- Capabilities
- sparse-attention, long-context-modeling, density-estimation
- Integrations
- Use Cases
- long-sequence-generation, image-generation, audio-generation
- API Available
- No
- Tags
- sparse-attention, long-context, transformers, generative-modeling, openai
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
61.2Adoption
68
Quality
85
Freshness
38
Citations
68
Engagement
0
Put AI to work for your business
Deploy this paper alongside autonomous AaaS agents that handle tasks end-to-end — no babysitting required.