arXiv:2508.19982cs.CLcs.AI2025-08被引 67

扩散语言模型提前识别答案,可大幅缩短生成时间。

Diffusion Language Models Know the Answer Before Decoding

  • 发现扩散模型在半数步内已能准确预测答案,无需完整迭代
  • 实验显示在GSM8K和MMLU上仅用一半步数即可达97%和99%准确率
  • 提出Prophet方法,通过置信度差值判断是否提前终止,无需训练

扩散语言模型(DLMs)虽可并行生成且支持灵活词序,但推理速度仍落后于自回归模型,主要因双向注意力开销及高质量输出需大量精炼步骤。本文揭示并利用一个被忽视的特性:在许多情况下,正确答案在最终解码前半步即可被内部识别,无论采用半自回归还是随机掩码策略。例如,在GSM8K和MMLU上,分别有高达97%和99%的实例仅用一半精炼步数即可正确解码。基于此,我们提出Prophet——一种无需训练的快速解码范式,通过顶部两个候选答案的置信度差值动态决定是否继续精炼或“全押”(一步解码剩余全部词元)。该方法无缝集成于现有DLM实现中,开销极小,无需额外训练。在LLaDA-8B与Dream-7B上的多任务评估表明,Prophet最多将解码步数减少3.4倍,同时保持高生成质量。结果表明,将DLM解码视为何时停止采样的问题,早期收敛为加速推理提供了简单而有效的机制,可与现有提速技术互补。代码已开源:https://github.com/pixeli99/Prophet。

原文摘要 · Abstract (English)

Diffusion language models (DLMs) have recently emerged as an alternative to autoregressive approaches, offering parallel sequence generation and flexible token orders. However, their inference remains slower than that of autoregressive models, primarily due to the cost of bidirectional attention and the large number of refinement steps required for high quality outputs. In this work, we highlight and leverage an overlooked property of DLMs early answer convergence: in many cases, the correct answer can be internally identified by half steps before the final decoding step, both under semi-autoregressive and random remasking schedules. For example, on GSM8K and MMLU, up to 97% and 99% of instances, respectively, can be decoded correctly using only half of the refinement steps. Building on this observation, we introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding. Specifically, Prophet dynamically decides whether to continue refinement or to go "all-in" (i.e., decode all remaining tokens in one step), using the confidence gap between the top-2 prediction candidates as the criterion. It integrates seamlessly into existing DLM implementations, incurs negligible overhead, and requires no additional training. Empirical evaluations of LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the number of decoding steps by up to 3.4x while preserving high generation quality. These results recast DLM decoding as a problem of when to stop sampling, and demonstrate that early decode convergence provides a simple yet powerful mechanism for accelerating DLM inference, complementary to existing speedup techniques. Our code is publicly available at https://github.com/pixeli99/Prophet.

扩散模型加速推理文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。