Fast Inference from Transformers via Speculative Decoding
by Google Research · free · Last verified 2026-03-17
Introduced speculative decoding, a lossless inference acceleration technique that uses a smaller, faster draft model to propose multiple tokens, then verifies them in parallel with the target model in a single forward pass. This achieves 2-3x speedup without any degradation in output quality or distribution.
https://arxiv.org/abs/2211.17192 ↗B+
B+—Good
Adoption: AQuality: A+Freshness: B+Citations: AEngagement: F
Specifications
- License
- Open Access
- Pricing
- free
- Capabilities
- inference-acceleration, lossless-decoding, parallel-verification
- Integrations
- Use Cases
- llm-inference-optimization, latency-reduction, production-serving
- API Available
- No
- Tags
- speculative-decoding, inference-efficiency, draft-model, lossless-acceleration, llm-inference
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
74.7Adoption
88
Quality
95
Freshness
74
Citations
82
Engagement
0
Put AI to work for your business
Deploy this paper alongside autonomous AaaS agents that handle tasks end-to-end — no babysitting required.