arXiv:2603.25702cs.CL2026-03被引 3

无需训练的自推测解码,让扩散语言模型更快更准。

S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation

  • 用同一模型在小块内自回归验证,实现并行生成与局部纠错结合
  • 在三类主流模型上最高提速4.7倍,精度提升4.5点
  • 适合追求高速高质生成的部署场景,尤其适配已有扩散模型

块扩散语言模型通过块级自回归解码与块内并行去噪,为超越自回归生成提供了可能。但在实际加速所需的少步数情况下,标准置信度阈值解码常不稳定:激进阈值损害质量,保守阈值需额外去噪步骤。现有方法要么需额外训练,要么增加推理时计算开销。我们提出S2D2,一种面向块扩散语言模型的无训练自推测解码框架。核心观察是:当块大小缩减至1时,块扩散模型退化为自回归模型,使同一预训练模型可同时充当起草者与验证者。S2D2在标准块扩散解码中插入推测性验证步骤,并采用轻量路由策略决定验证是否值得成本。这形成混合解码轨迹:扩散并行提议词元,自回归模式作为局部序列级评判者。在三大主流块扩散模型族上,S2D2始终优于强基线置信度阈值方法。在SDAR上,相比自回归解码最高提速4.7倍,相比调优动态解码基线提速1.57倍,且精度提升最高达4.5点。在LLaDA2.1-Mini上,S2D2与内置自纠正机制互补,保守设置下比静态基线快4.4倍且精度略高。

原文摘要 · Abstract (English)

Block-diffusion language models offer a promising path toward faster-than-autoregressive generation by combining block-wise autoregressive decoding with within-block parallel denoising. However, in the few-step regime needed for practical acceleration, standard confidence-thresholded decoding is often brittle: aggressive thresholds hurt quality, while conservative thresholds require unnecessary denoising steps. Existing approaches that address this issue either require additional training or incur extra test-time compute. We present S2D2, a training-free self-speculative decoding framework for block-diffusion language models. Our key observation is that a block-diffusion model becomes autoregressive when the block size is reduced to one, allowing the same pretrained model to act as both drafter and verifier. S2D2 inserts a speculative verification step into standard block-diffusion decoding and uses lightweight routing policies to decide when verification is worth its cost. This yields a hybrid decoding trajectory in which diffusion proposes tokens in parallel, while the autoregressive mode acts as a local sequence-level critic. Across three mainstream block-diffusion families, S2D2 consistently improves the accuracy-speed tradeoff over strong confidence-thresholding baselines. On SDAR, we observe up to $4.7\times$ speedup over autoregressive decoding, and up to $1.57\times$ over a tuned dynamic decoding baseline while improving accuracy by up to $4.5$ points. On LLaDA2.1-Mini, S2D2 remains complementary to built-in self-correction, including a conservative setting where it is $4.4\times$ faster than the static baseline with slightly higher accuracy.

扩散模型快速生成自推测语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。