用不确定性量化提升掩码扩散模型的生成质量
Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty
- 引入去噪熵衡量生成路径中的累积不确定性
- 两种优化策略使复杂任务准确率显著提升
- 适合研究生成过程可控性的学者参考
掩码扩散模型（MDMs）提供灵活的非自回归生成，但其自由度也带来挑战：最终输出质量高度依赖解码顺序。我们首次将输出质量波动归因于生成路径上的累积预测不确定性。为此，提出可计算的去噪熵作为内部信号，评估生成过程。基于该指标，设计两种路径优化算法：事后选择方法与实时引导策略。实验表明，熵引导方法在复杂推理、规划和代码生成基准上持续提升准确率。本工作确立去噪熵为理解与控制生成过程的理论工具，将MDMs中的不确定性从缺陷转化为发现高质量解的关键优势。
原文摘要 · Abstract (English)
Masked Diffusion Models (MDMs) offer flexible, non-autoregressive generation, but this freedom introduces a challenge: final output quality is highly sensitive to the decoding order. We are the first to formalize this issue, attributing the variability in output quality to the cumulative predictive uncertainty along a generative path. To quantify this uncertainty, we introduce Denoising Entropy, a computable metric that serves as an internal signal for evaluating generative process. Leveraging this metric, we propose two algorithms designed to optimize the decoding path: a post-hoc selection method and a real-time guidance strategy. Experiments demonstrate that our entropy-guided methods significantly improve generation quality, consistently boosting accuracy on challenging reasoning, planning, and code benchmarks. Our work establishes Denoising Entropy as a principled tool for understanding and controlling generation, effectively turning the uncertainty in MDMs from a liability into a key advantage for discovering high-quality solutions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。