arXiv:2512.20573cs.LGcs.AI2025-12被引 4

用扩散模型加速大模型推理,快且不丢质量

Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs

  • 用扩散模型并行生成候选文本,动态调整猜测长度
  • 无需微调,最高提速4.9倍,单次可猜70个词
  • 适合追求高吞吐的部署场景,开源可用

扩散型大语言模型(dLLMs)能实现快速并行生成,但单独使用时存在效率与质量的权衡。本文发现,若合理应用,dLLMs的并行特性反而可成为推测解码中生成器的优势:其高速生成大幅降低被拒绝的风险,从而实现长序列推测,带来显著加速。为此提出FailFast框架,通过动态调整推测长度,在难预测区域快速失败以减少延迟,在易预测区域激进扩展推测长度以降低验证延迟(部分情况可一次性推测并接受70个token)。无需任何微调,FailFast实现了对自回归模型的无损加速,相比原生解码最高提速4.9倍,较最优朴素dLLM生成器提速1.7倍,较EAGLE-3提速1.7倍,适用于多种模型与负载。项目已开源于https://github.com/ruipeterpan/failfast。

原文摘要 · Abstract (English)

Diffusion Large Language Models (dLLMs) offer fast, parallel token generation, but their standalone use is plagued by an inherent efficiency-quality tradeoff. We show that, if carefully applied, the attributes of dLLMs can actually be a strength for drafters in speculative decoding with autoregressive (AR) verifiers. Our core insight is that dLLM's speed from parallel decoding drastically lowers the risk of costly rejections, providing a practical mechanism to effectively realize the (elusive) lengthy drafts that lead to large speedups with speculative decoding. We present FailFast, a dLLM-based speculative decoding framework that realizes this approach by dynamically adapting its speculation length. It "fails fast" by spending minimal compute in hard-to-speculate regions to shrink speculation latency and "wins big" by aggressively extending draft lengths in easier regions to reduce verification latency (in many cases, speculating and accepting 70 tokens at a time!). Without any fine-tuning, FailFast delivers lossless acceleration of AR LLMs and achieves up to 4.9$\times$ speedup over vanilla decoding, 1.7$\times$ over the best naive dLLM drafter, and 1.7$\times$ over EAGLE-3 across diverse models and workloads. We open-source FailFast at https://github.com/ruipeterpan/failfast.

大模型推理扩散模型推测解码加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。