arXiv:2510.09544cs.CL2025-10被引 3

揭示扩散型大模型真实推理能力,发现并缓解并行与因果推理的矛盾

Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models

  • 提出并行-顺序矛盾(PSC)概念,解释扩散模型推理受限根源
  • 实验证明高难度任务下模型退化为自回归行为,提示词加剧步骤数翻倍
  • 给出并行提示、早期停止等实用方案,提升推理效率与深度

最近,扩散型大语言模型(DLLMs)因其高吞吐量和有效的序列推理能力,成为自回归模型(ALLMs)的有力替代。然而,支持同时更新多个词元的并行解码,与严格因果顺序推理存在根本冲突。我们首次将此冲突识别为核心的并行-顺序矛盾(PSC)。在简单与复杂推理任务中的行为分析表明,仅当输出可直接判定时,DLLMs才真正实现并行;任务难度上升后,模型会退化为类似自回归的行为。此外,自回归提示显著增加解码步骤近一倍,且未提升质量,进一步放大了PSC的影响。该矛盾还限制了模型的自我反思、推理深度与探索广度。为此,我们引入三个扩展维度:并行、扩散与顺序。实证显示,仅并行扩展能持续带来改进,而扩散与顺序扩展受制于PSC。基于上述发现,我们提出三种实用缓解策略:面向并行的提示设计、扩散过程早期停止与并行扩展,有效降低因PSC导致的无效性与低效问题。

原文摘要 · Abstract (English)

Recently, Diffusion Large Language Models (DLLMs) have offered high throughput and effective sequential reasoning, making them a competitive alternative to autoregressive LLMs (ALLMs). However, parallel decoding, which enables simultaneous token updates, conflicts with the causal order often required for rigorous reasoning. We first identify this conflict as the core Parallel-Sequential Contradiction (PSC). Behavioral analyses in both simple and complex reasoning tasks show that DLLMs exhibit genuine parallelism only for directly decidable outputs. As task difficulty increases, they revert to autoregressive-like behavior, a limitation exacerbated by autoregressive prompting, which nearly doubles the number of decoding steps with remasking without improving quality. Moreover, PSC restricts DLLMs' self-reflection, reasoning depth, and exploratory breadth. To further characterize PSC, we introduce three scaling dimensions for DLLMs: parallel, diffusion, and sequential. Empirically, while parallel scaling yields consistent improvements, diffusion and sequential scaling are constrained by PSC. Based on these findings, we propose several practical mitigations, parallel-oriented prompting, diffusion early stopping, and parallel scaling, to reduce PSC-induced ineffectiveness and inefficiencies.

扩散模型推理能力语言模型并行解码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。