PaperLLMsv1.0

Generating Long Sequences with Sparse Transformers

by OpenAI · free · Last verified 2026-03-17

The Sparse Transformer introduces factored sparse attention patterns to reduce the self-attention mechanism's complexity from O(n²) to O(n√n). This innovation enables Transformer models to process and generate sequences thousands of steps long, making them effective for high-resolution generative tasks.

https://arxiv.org/abs/1904.10509 ↗

B—Above Average

Adoption: BQuality: AFreshness: DCitations: BEngagement: F

Specifications

License: Open Access
Pricing: free
Capabilities: sparse-attention-mechanisms, long-context-modeling, autoregressive-density-estimation, generative-modeling-of-long-sequences, high-resolution-image-generation, long-form-text-generation, raw-audio-synthesis, reduced-computational-complexity, reduced-memory-footprint
Integrations
Use Cases: [object Object], [object Object], [object Object], [object Object], [object Object]
API Available: No
Tags: sparse-attention, long-context, transformers, generative-modeling, openai, efficient-transformers, computational-complexity, density-estimation, image-generation, audio-generation
Added: 2026-03-17
Completeness: 0.9%

Index Score

61.2

Adoption

Quality

Freshness

Citations

Engagement

Need this tool deployed for your team?

Get a Custom Setup

Explore the full AI ecosystem on Agents as a Service