提出进化解码机制,解决扩散模型数学推理中的信心陷阱问题。
Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs

- 将解码视为进化过程,结合选择与变异策略优化推理路径。
- 在多个基准上使LLaDA 2.0的数学推理正确率显著提升。
- 无需训练,适合提升现有扩散型大模型的逻辑推理能力。
扩散语言模型(dLLMs)作为自回归模型的替代方案,通过逐块逐步去掩码实现高效生成。然而,其强大的通用性能并未转化为可靠的数学推理能力,因为正确性依赖于连贯的数值-符号推理轨迹。本文分析了LLaDA 2.0的解码轨迹,发现存在一种重复出现的扩散信心陷阱:局部标记置信度与全局推理正确性脱节。分析揭示两种典型失败模式:采样敏感型失败(存在正确路径但不稳定),以及采样一致型失败(重复采样收敛至高置信但错误的循环)。为此,我们提出进化解码(Evolutionary Decoding),一种无需训练的测试时扩展框架,将扩散解码视为候选推理状态的演化过程。该框架结合逐步选择(保留有用数值符号信号,抑制重复模式)与块级变异(引入结构化替代路径以逃离错误高置信区域)。实验表明,进化解码在多个基准上优于基于置信度的解码策略,显著提升了LLaDA 2.0的数学推理可靠性。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) have emerged as a promising alternative to autoregressive LLMs, offering efficient generation through block-wise progressive unmasking. However, their strong general-purpose performance does not necessarily translate into reliable mathematical reasoning, where correctness depends on preserving coherent numerical-symbolic reasoning trajectories. In this work, we analyze the decoding trajectories of LLaDA 2.0 and identify a recurring diffusion confidence trap: local token confidence can become misaligned with global reasoning correctness during progressive block decoding. Our analysis reveals two representative failure regimes: sampling-sensitive failures, where correct paths exist but are unstable, and sampling-consistent failures, where repeated sampling converges to repetitive high-confidence but incorrect continuations. Motivated by this observation, we propose Evolutionary Decoding, a training-free test-time scaling framework that views diffusion decoding as an evolutionary process over candidate reasoning states. The framework combines step-wise selection, which preserves useful numerical-symbolic signals and suppresses repetitive patterns, with block-wise mutation, which introduces structured alternatives to escape incorrect high-confidence basins. Experiments on multiple benchmarks show that Evolutionary Decoding improves LLaDA 2.0 over confidence-based decoding, leading to more reliable mathematical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。