用单次前向传播完成多步草案生成，显著提速大模型推理
FastEagle: Cascaded Drafting for Accelerating Speculative Decoding
- 采用轻量级层叠结构替代传统逐词生成，实现一次前向传播出完整草案
- 在多个模型和任务上速度提升超EAGLE-3，平均采纳长度相当
- 适合追求高吞吐、低延迟的部署场景，尤其适合实时生成应用
推测解码通过并行生成候选文本并验证来加速生成，但当前最优草案生成器（如EAGLE）仍需N次串行计算生成N个词。我们提出FastEagle，一种非自回归的级联草案生成器，可在单次前向传播中输出完整草案。FastEagle以轻量级层叠结构替代时间步，通过分层监督训练缓解误差累积。结合约束草案树结构，保持无损验证开销。在多个LLM（Vicuna-13B、LLaMA-Instruct 3.x、DeepSeek-R1-Distill-LLaMA）和任务（MT-Bench、HumanEval、GSM8K、CNN/DM、Alpaca）上，FastEagle在贪婪与随机解码下均持续优于EAGLE-3的速度表现，且平均采纳长度相当。结果表明，消除草案生成中的串行依赖是实现无损大模型推理加速的可行路径。
原文摘要 · Abstract (English)
Speculative decoding accelerates generation by drafting candidates and verifying them in parallel, yet state-of-the-art drafters (e.g., EAGLE) still require N sequential passes to propose N tokens. We present FastEagle, a non-autoregressive cascaded drafter that emits an entire draft in a single forward pass. FastEagle replaces temporal steps with a lightweight layer cascade and trains with layer-wise supervision to mitigate error accumulation. Coupled with a constrained draft tree that preserves lossless verification cost, FastEagle delivers substantial wall-clock speedups over strong autoregressive drafters while maintaining competitive acceptance behavior. Across multiple LLMs (Vicuna-13B, LLaMA-Instruct 3.x, and DeepSeek-R1-Distill-LLaMA) and tasks (MT-Bench, HumanEval, GSM8K, CNN/DM, Alpaca), FastEagle consistently outperforms EAGLE-3 in speedup under both greedy and stochastic decoding, with comparable average acceptance lengths. These results indicate that removing sequential dependencies in drafting is a practical path toward lossless LLM inference acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。