arXiv:2608.08721cs.CLcs.AI2026-08

用动态优化提升扩散模型的并行推理速度,显著加快大模型生成。

LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization

  • 基于边际收益准则动态决定每轮推测长度,平衡生成与验证开销。
  • 在多个模型和任务上实现0.5至1.5倍性能提升,最高达8.49倍加速。
  • 无需训练、即插即用,适合希望提速的扩散模型推理场景。

推测解码通过并行生成多个候选词来加速大语言模型推理,其效率关键取决于每轮解码中选择的推测长度。现有动态推测方法通过估计可接受的词数来决定长度,这适用于逐个生成词的自回归式推测器。然而,近期出现的扩散模型推测器能以极低开销并行生成候选词块,使得核心问题从‘生成多少词’转变为‘哪些生成词值得验证’。因此,我们将动态推测长度选择重新建模为期望加速优化,并推导出一种边际准则:仅当接受收益超过额外验证成本时才扩展推测序列。基于此准则,我们提出LibraSpec,一种无需训练、即插即用的算法,通过迭代使用推测器置信度评分确定推测长度。理论证明该算法单调收敛至最优推测长度。在六种目标模型、三种扩散式推测解码方法及数学、编程、对话基准上的实验表明,无论在贪婪还是采样设置下均取得一致提升,相比基线进一步提高0.5~1.5倍性能,最高可达自回归解码的8.49倍加速。

原文摘要 · Abstract (English)

Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round. Existing dynamic speculation methods select the speculation length by estimating how many tokens will be accepted, which is reasonable for autoregressive drafters that generates tokens sequentially. The recent wave of diffusion-based drafters, however, generates candidate blocks in parallel at substantially lower drafting cost, shifting the key question from how many tokens to generate to how many generated tokens are worth verifying. We therefore reformulate dynamic speculative-length selection as expected-speedup optimization and derive a marginal criterion that extends the speculative sequence only when its acceptance gain outweighs the additional verification cost. Building on this criterion, we develop \textit{LibraSpec}, a training-free and plug-and-play algorithm that iteratively determines the speculative length using drafter confidence scores. Theoretically, we prove that LibraSpec monotonically converges toward the optimal speculative length. Experiments across six target models, three diffusion-based speculative decoding methods, and math, coding, and chat benchmarks show consistent improvements under both greedy and sampling settings, achieving a further $0.5\sim1.5\times$ improvement over baselines and up to $8.49\times$ speedup over autoregressive decoding.

推理加速扩散模型推测解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。