arXiv:2602.22871cs.CLcs.AI2026-02被引 1

用扩散模型生成多种推理路径,再拼接最优步骤提升数学与编程题准确率。

Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

  • 通过扩散模型生成多条低成本推理路径,保留中间步骤
  • 用奖励模型评分并拼接高质量步骤,最终由自回归模型输出答案
  • 无需训练,比传统方法快1.8倍,难题上准确率提升23.8%

大语言模型推理常受益于生成多个思维链,但现有聚合策略多为轨迹级(如选择最佳路径或投票),忽视了部分正确路径中的有用中间结果。我们提出Stitching Noisy Diffusion Thoughts,一种自一致性框架,将廉价采样的扩散推理转化为可复用的步骤级候选池。给定问题后,(i) 使用掩码扩散语言模型采样大量多样化、低成本的推理路径;(ii) 用现成的过程奖励模型(PRM)对每个中间步骤评分;(iii) 将各路径中质量最高的步骤拼接成复合推理链,再由自回归(AR)模型(求解器)仅重新计算最终答案。该模块化流程分离探索(扩散)、评估与求解合成,避免单体统一架构的同时保持广泛搜索。在数学推理基准上,步骤级重组在难题上效果最佳,消融实验表明最终的AR求解器对将拼接但不完美的推理链转化为准确答案至关重要。采用低置信度扩散采样与并行独立推演,本训练免费框架在六项数学与编程任务上平均准确率提升达23.8%,同时相较传统扩散模型(如Dream、LLaDA)和统一架构(如TiDAR)实现最高1.8倍的延迟降低。代码已开源:https://github.com/roymiles/diffusion-stitching。

原文摘要 · Abstract (English)

Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from partial or "nearly correct" attempts. We propose Stitching Noisy Diffusion Thoughts, a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates. Given a problem, we (i) sample many diverse, low-cost reasoning trajectories using a masked diffusion language model, (ii) score every intermediate step with an off-the-shelf process reward model (PRM), and (iii) stitch these highest-quality steps across trajectories into a composite rationale. This rationale then conditions an autoregressive (AR) model (solver) to recompute only the final answer. This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search. Across math reasoning benchmarks, we find that step-level recombination is most beneficial on harder problems, and ablations highlight the importance of the final AR solver in converting stitched but imperfect rationales into accurate answers. Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks. At the same time, it achieves up to a 1.8x latency reduction relative to both traditional diffusion models (e.g., Dream, LLaDA) and unified architectures (e.g., TiDAR). Code is available at https://github.com/roymiles/diffusion-stitching.

推理增强扩散模型自一致性数学推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。