用非自回归模型快速生成推理步骤，再让自回归模型精炼答案。
Parallel Thinking, Sequential Answering: Bridging NAR and AR for Efficient Reasoning
- 先用并行生成的NAR模型快速产出中间推理过程
- 再用序列生成的AR模型基于推理步骤给出精确答案，提升26%性能
- 适合需要高效且准确推理的任务，如数学和代码生成
我们研究通过整合自回归（AR）与非自回归（NAR）语言模型来提升推理任务表现。AR模型虽能生成连贯文本，但推理速度慢，尤其在数学、编程等需长链推理的领域；而NAR模型（如离散扩散模型）支持并行生成，显著加速，但通常牺牲输出质量。为此，我们提出新范式：由NAR模型高效生成中间推理轨迹，再由AR模型据此生成精准最终答案。实验表明，该方法相较强基线提升26%，同时大幅降低推理开销。
原文摘要 · Abstract (English)
We study reasoning tasks through a framework that integrates auto-regressive (AR) and non-autoregressive (NAR) language models. AR models, which generate text sequentially, excel at producing coherent outputs but often suffer from slow inference, particularly in reasoning-intensive domains such as mathematics and code, where lengthy chains of thought are required. In contrast, NAR models, such as discrete diffusion models, allow parallel generation and offer substantial speedups, though typically at the cost of reduced output quality. To address these limitations, we introduce a new paradigm in which an NAR model efficiently produces intermediate reasoning traces, which subsequently guide an AR model to deliver precise final answers. Experiments demonstrate that our approach yields significant 26% improvements over strong baselines while substantially reducing inference cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。