让扩散语言模型可回退，解决生成卡死问题。
Reversible Diffusion Decoding for Diffusion Language Models
- 引入可逆解码机制，检测并回溯停滞状态。
- 通过缓存状态实现无重新计算的高效回退。
- 适合追求生成稳定性和效率的场景。
扩散语言模型通过分块并行生成实现快速文本生成，但其不可逆性可能导致在次优上下文中停滞，即反向扩散过程无法继续推进。本文提出可逆扩散解码（RDD），在分块扩散生成中引入可逆性。RDD 将停滞识别为反向过程的状态依赖失败，并通过缓存模型状态实现无需重新计算的高效回溯。为避免重复陷入失败路径，RDD 采用置信度引导的重新掩码策略，仅对不确定的词元进行选择性重初始化，同时保留可靠的上下文。该可逆框架使解码能从早期错误承诺中恢复，同时保持扩散生成的并行效率。实验表明，与基线相比，RDD 在几乎不增加计算开销的情况下提升了生成鲁棒性和质量。
原文摘要 · Abstract (English)
Diffusion language models enable parallel token generation through block-wise decoding, but their irreversible commitments can lead to stagnation, where the reverse diffusion process fails to make further progress under a suboptimal context.We propose Reversible Diffusion Decoding (RDD), a decoding framework that introduces reversibility into block-wise diffusion generation. RDD detects stagnation as a state-dependent failure of the reverse process and enables efficient backtracking to earlier blocks without recomputation via cached model states. To avoid repeated failure trajectories, RDD applies confidence-guided re-masking to selectively reinitialize uncertain tokens while preserving reliable context.This reversible formulation allows decoding to recover from early commitment errors while maintaining the parallel efficiency of diffusion-based generation. Experiments show that RDD improves generation robustness and quality over baselines with minimal computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。