用扩散模型提速大模型推理，不降精度还快5.5倍
SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding
- 用非自回归扩散模型做草稿，突破传统依赖瓶颈
- 实现平均55%的吞吐提升，最高达5.5倍加速
- 适合追求高效推理的工业级LLM部署场景
推测解码已成为加速大语言模型推理的标准方法，通过无损的草稿-验证流程绕过自回归生成的延迟，取得显著加速效果。然而现有方法仍受两大瓶颈制约：(1) 草稿阶段存在自回归依赖，限制并行性；(2) 草稿与验证模型不匹配导致频繁拒绝。本文提出SpecDiff-2，利用离散扩散模型作为非自回归草稿器解决第一个瓶颈，并设计新方法校准扩散草稿器与自回归验证器之间的对齐关系，解决第二个瓶颈。在涵盖推理、编码和数学任务的综合基准测试中，SpecDiff-2达到新SOTA表现，相比此前基线平均提升55%的每秒生成词数，相比标准解码实现最高5.5倍平均加速，且无准确率损失。
原文摘要 · Abstract (English)
Speculative decoding has become the standard approach for accelerating Large Language Model (LLM) inference. It exploits a lossless draft-then-verify procedure to circumvent the latency of autoregressive decoding, achieving impressive speed-ups. Yet, current speculative decoding approaches remain limited by two fundamental bottlenecks: (1) the autoregressive dependency during drafting which limits parallelism, and (2) frequent rejections of draft tokens caused by misalignment between the draft and verify models. This paper proposes SpecDiff-2, a novel framework to jointly address these two bottlenecks. It leverages discrete diffusion as a non-autoregressive drafter to address bottleneck (1) and develops novel techniques to calibrate discrete diffusion drafters with autoregressive verifiers, addressing bottleneck (2). Experimental results across a comprehensive benchmark suite show that SpecDiff-2 achieves a new state-of-the-art across reasoning, coding, and mathematical benchmarks, improving tokens-per-second by up to an average of +55% over previous baselines and obtaining up to 5.5x average speed-up over standard decoding, without any loss of accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。