用扩散模型生成草稿，大幅提升大模型推理速度。
DEER: Draft with Diffusion, Verify with Autoregressive Models
- 用扩散模型作草稿生成，可并行处理、避免逐词累积误差
- 单步生成32个词草稿，接受率远超传统方法的10个词
- 适合追求高推理效率的AI系统开发者
大语言模型驱动的智能体与推理系统面临效率瓶颈，主要源于自回归解码带来的固有延迟。推测解码通过草稿-验证机制缓解该问题，但现有方法依赖自回归草稿模型（drafter），存在两大根本缺陷：（1）逐步积累的不确定性导致目标模型与草稿模型间的信任持续衰减；（2）自回归草稿模型本质为串行解码。两者共同限制了加速效果。本文提出，基于扩散的大型语言模型（dLLM）作为草稿模型，因其不同的概率建模方式和高效的并行解码策略，可天然克服上述问题。基于此，我们构建DEER框架，采用扩散模型生成草稿、自回归模型进行验证。为保证草稿质量，DEER设计两阶段训练流程以对齐dLLM草稿模型与目标AR模型，并采用单步解码生成长草稿段落。实验表明，DEER可实现最高达32词的草稿接受长度，显著超越EAGLE-3的10词上限。在HumanEval评测中，使用Qwen3-30B-A3B模型时，DEER达到5.54倍加速，而EAGLE-3仅为2.41倍。代码、模型及演示将公开于https://czc726.github.io/DEER/
原文摘要 · Abstract (English)
Efficiency, as a critical practical challenge for LLM-driven agentic and reasoning systems, is increasingly constrained by the inherent latency of autoregressive (AR) decoding. Speculative decoding mitigates this cost through a draft-verify scheme, yet existing approaches rely on AR draft models (a.k.a., drafters), which introduce two fundamental issues: (1) step-wise uncertainty accumulation leads to a progressive collapse of trust between the target model and the drafter, and (2) inherently sequential decoding of AR drafters. Together, these factors cause limited speedups. In this paper, we show that a diffusion large language model (dLLM) drafters can naturally overcome these issues through its fundamentally different probabilistic modeling and efficient parallel decoding strategy. Building on this insight, we introduce DEER, an efficient speculative decoding framework that drafts with diffusion and verifies with AR models. To enable high-quality drafting, DEER employs a two-stage training pipeline to align the dLLM-based drafters with the target AR model, and further adopts single-step decoding to generate long draft segments. Experiments show DEER reaches draft acceptance lengths of up to 32 tokens, far surpassing the 10 tokens achieved by EAGLE-3. Moreover, on HumanEval with Qwen3-30B-A3B, DEER attains a 5.54x speedup, while EAGLE-3 achieves only 2.41x. Code, model, demo, etc, will be available at https://czc726.github.io/DEER/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。