arXiv:2603.04514cs.AI2026-03

通过动态调控生成过程,让不同词元按需迭代,显著提速文本生成。

Progressive Refinement Regulation for Accelerating Diffusion Language Model Decoding

  • 基于完整生成轨迹判断词元收敛进度,实现自适应调控。
  • 在不损失质量前提下,解码速度提升显著,实测加速效果明显。
  • 适合追求高效生成的场景,尤其适用于扩散语言模型优化。

扩散语言模型通过统一的迭代去噪规则生成文本,但实际中各词元收敛速度差异大,导致大量冗余迭代。现有方法依赖固定解码过程中的瞬时信号判断是否需要精炼,而本文提出一种动态的渐进式精炼调控框架(PRR),基于完整生成轨迹推导出词元级的实证收敛进度信号。在此基础上,PRR通过轻量级逐词控制器,在渐进自演化训练下以温度调节分布形状来控制精炼行为。实验表明,该方法在保持生成质量的同时显著加速了扩散语言模型的解码过程。

原文摘要 · Abstract (English)

Diffusion language models generate text through iterative denoising under a uniform refinement rule applied to all tokens. However, tokens stabilize at different rates in practice, leading to substantial redundant refinement and motivating refinement control over the denoising process. Existing approaches typically assess refinement necessity from instantaneous, step-level signals under a fixed decoding process. In contrast, whether a token has converged is defined by how its prediction changes along its future refinement trajectory. Moreover, changing the refinement rule reshapes future refinement trajectories, which in turn determine how refinement rules should be formulated, making refinement control inherently dynamic. We propose \emph{Progressive Refinement Regulation} (PRR), a progressive, trajectory-grounded refinement control framework that derives a token-level notion of empirical convergence progress from full decoding rollouts. Based on this signal, PRR learns a lightweight token-wise controller to regulate refinement via temperature-based distribution shaping under a progressive self-evolving training scheme. Experiments show that PRR substantially accelerates diffusion language model decoding while preserving generation quality.

扩散模型生成加速语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。