发现掩码扩散模型推理时会因自信判断导致错误，训练方式加剧问题。
The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
- 用置信度决定生成顺序，忽视长距离依赖关系。
- 在多位数加法任务中，错误率比随机掩码高十倍。
- 适合关注复杂推理鲁棒性的研究者阅读。
掩码扩散语言模型（MDMs）支持任意顺序生成，当前普遍采用基于置信度的解码策略。为优化此策略，近期训练方法试图让训练掩码模式与生成时观察到的模式对齐。然而，我们指出置信度解码本质上与复杂推理所需的逻辑轨迹不一致，而置信度对齐的训练反而强化了这种偏差。通过多位数加法任务验证：该策略会过早预测局部简单数字，忽略长程依赖，导致在困难输入上产生高置信度错误。相较于传统随机掩码，置信度对齐训练使错误率上升一个数量级。在五个不同推理任务中均出现类似现象：置信度解码引发复杂输入下的失败，而对齐训练进一步放大问题。相反，看似低效的随机掩码却能稳健保持解决难题所需的推理轨迹条件性。
原文摘要 · Abstract (English)
Masked diffusion language models (MDMs) uniquely support any-order generation, with confidence-based decoding currently serving as the de facto standard inference policy. To optimize for this, recent training schemes attempt to align training mask patterns directly with those observed during generation. However, we argue that confidence-based decoding is inherently misaligned with the logical-flow trajectories required for complex reasoning, and that confidence-aligned training actively entrenches this misalignment. We make this concrete using multi-digit addition, where the decoding strategy prematurely predicts locally easy digits before resolving their long-range dependencies, producing high-confidence errors on challenging inputs. While traditional random masking keeps the failure rate low on this challenging tail, confidence-aligned training amplifies the error rate by an order of magnitude. Across five distinct reasoning tasks, this same pattern emerges with task-dependent severity: confidence-based decoding induces failures on highly complex inputs, and confidence-aligned training exacerbates them. In contrast, random masking -- despite its perceived inefficiency -- robustly preserves the reasoning-trajectory conditionals essential for solving the challenging tail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。