让大模型在长推理中反复回顾关键思路,提升解题准确率。
Stateful Reasoning via Insight Replay
- 定期提取并重播推理过程中的关键洞察,保持其可访问性。
- 在24个实验设置中平均提升1.65分,最高达9.2分。
- 适合需要长序列推理的数学与代码任务,尤其对大模型有效。
链式思维(CoT)是激发大模型多步推理能力的基础,但研究表明其效果并非随推理长度单调提升:虽然更长的CoT有助于解决更难问题,但在特定问题上,准确率随长度增加到一定点后反而下降。我们发现主要原因在于:随着推理链增长,模型对早期生成的关键洞察注意力逐渐减弱,导致这些关键信息在最需要时难以调用。为此,我们提出状态化推理方法InsightReplay,即周期性从推理轨迹中提取关键洞察,并在其附近重播,使其在长推理过程中持续可及。在覆盖模型规模{8B, 30B}、模型族{Qwen3.5, DeepSeek-R1-Distill-Qwen, Gemma-4}、推理基准{AIME, HMMT, GPQA Diamond, LiveCodeBench v5}的2×3×4基准网格上进行大量实验,结果显示,3轮InsightReplay在全部24个设置中均取得准确率提升,平均优于标准CoT 1.65分,单设点最高提升9.2分(在R1-Distill-32B的LiveCodeBench v5子集上)。结果表明,测试时扩展的有效性不仅取决于推理量,更取决于关键中间洞察是否在整个推理轨迹中始终可访问。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models, but recent studies show that its benefits do not scale monotonically with chain length: while longer CoT generally enables a model to tackle harder problems, on a given problem, accuracy typically increases with CoT length up to a point, after which it declines. We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens, making those insights progressively less accessible when they are most needed. Therefore, we propose \textbf{InsightReplay}, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier, keeping them accessible as the reasoning scales. Extensive experiments on a $\mathbf{2}\!\times\!\mathbf{3}\!\times\!\mathbf{4}$ benchmark grid, covering model scales $\{\text{8B}, \text{30B}\}$, model families $\{\text{Qwen3.5}, \text{DeepSeek-R1-Distill-Qwen}, \text{Gemma-4}\}$, and reasoning benchmarks $\{\text{AIME}, \text{HMMT}, \text{GPQA Diamond}, \text{LiveCodeBench v5}\}$, show that 3-round InsightReplay yields accuracy gains across \textbf{all 24 settings}, with an averaged improvement of $\mathbf{+1.65}$ points over standard CoT, and a largest single-setting gain of $\mathbf{+9.2}$ points on R1-Distill-32B's LiveCodeBench v5 subset. Our results suggest that the effectiveness of test-time scaling depends not only on how much a model reasons, but also on whether critical intermediate insights remain accessible throughout long reasoning trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。