发现推测解码的缩放规律,实现推理加速2倍。
Scaling Laws for Speculative Decoding
- 提出三维度缩放定律,指导草稿模型效率优化。
- 实测性能超EAGLE2达1.5-2.2倍,峰值提升显著。
- 适用于大模型推理加速,尤其适合工业部署。
大型语言模型(LLM)在推理密集型架构中对高效解码的需求日益增长,如OpenAI-o3和DeepSeek-R1依赖长链式思维推理。本文研究基于密集LLM架构的推测解码技术,揭示其加速推理任务的基础规律。尽管并行草稿-验证循环的推测解码方法已被证明具有潜力,但其解码效率的缩放规律仍远未被充分探索,与传统预训练->SFT->RLHF范式下的骨干模型相比。本工作发现三条对数线性缩放定律(定理1.1、1.2、1.3),分别描述预训练词元量、草稿模型容量和解码批大小对草稿模型接受率(或解码速度)的影响。基于这些规律,我们构建了Scylla系统,协调多维度缩放以适配主流模型(Llama2/3、Qwen2.5)。实验验证显示,在温度T=0时,Scylla相比EAGLE2接受率提升1.5-2.2倍,相比EAGLE3提升0.3倍,且在摘要生成与问答任务上达到峰值性能(图2)。工业级推理引擎部署表明,其解码吞吐量较EAGLE2提升2倍(表5),验证了系统化缩放对高效LLM推理的变革潜力。代码将后续开源。
原文摘要 · Abstract (English)
The escalating demand for efficient decoding in large language models (LLMs) is particularly critical for reasoning-intensive architectures like OpenAI-o3 and DeepSeek-R1, which depend on extended chain-of-thought reasoning. This study investigates speculative decoding techniques through dense LLM architectures to establish foundational insights for accelerating reasoning tasks. While speculative decoding methods leveraging parallel draft-verification cycles have emerged as promising acceleration techniques, the scaling laws governing decoding efficiency remain under-explored compared to conventional backbone LLMs developed through Pretraining->SFT->RLHF training paradigms. In this work, we discover Log-linear Scaling Laws (Theorem 1.1, 1.2 and 1.3) governing draft model acceptance rate (or decoding speed) across three dimensions: pretraining token volume, draft model capacity, and decoding batch size. Building on these laws, we achieve Scylla, which coordinates multi-dimensional scaling for popular LLMs (Llama2/3, Qwen2.5). Empirical validation shows Scylla achieves 1.5-2.2 higher acceptance rate than EAGLE2 and 0.3 higher than EAGLE3 at temperature T = 0, with peak performance gains on summarization and QA tasks (Figure 2). Industrial inference engine deployments demonstrate 2X decoding throughput improvements over EAGLE2 (Table 5), validating the transformative potential of systematic scaling for efficient LLM inference. Code will be released later.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。