提出一种两阶段推理方法，提升大模型生成效率。
Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
- 用小模型先草稿，大模型再验证修正
- 相比传统逐字生成，推理速度显著提升
- 适合追求高效推理的部署场景
大规模语言模型（LLMs）的规模与复杂性不断增长，高效推理成为关键挑战。传统自回归解码因逐个生成标记而存在计算效率低下问题。推测解码通过两阶段框架——草稿与验证——解决此瓶颈：由小型高效模型生成初步草稿，再由大型精密模型进行修正。本文系统综述推测解码方法，将其分为以草稿为中心和以模型为中心两类，阐述各方法核心思想，强调其在扩展大模型推理中的潜力。本综述旨在为未来优化推测解码及其在真实大模型应用中的集成提供指导。
原文摘要 · Abstract (English)
Efficient inference in large language models (LLMs) has become a critical focus as their scale and complexity grow. Traditional autoregressive decoding, while effective, suffers from computational inefficiencies due to its sequential token generation process. Speculative decoding addresses this bottleneck by introducing a two-stage framework: drafting and verification. A smaller, efficient model generates a preliminary draft, which is then refined by a larger, more sophisticated model. This paper provides a comprehensive survey of speculative decoding methods, categorizing them into draft-centric and model-centric approaches. We discuss key ideas associated with each method, highlighting their potential for scaling LLM inference. This survey aims to guide future research in optimizing speculative decoding and its integration into real-world LLM applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。