发现大模型推理中信息流动断裂问题，提出修复方法提升准确率。
Reasoning Fails Where Step Flow Breaks
- 通过步间注意力图分析推理过程中的信息流动异常
- 揭示浅层过度关注当前步骤、深层逐渐丢失上下文两大问题
- 无需重训练即可提升多任务推理性能，适合模型优化研究者
大型推理模型（LRMs）在多步数学、科学和编程任务中表现良好，但其行为仍不稳定且难以解释，现有分析工具难以处理长而结构化的推理轨迹。本文提出 Step-Saliency，将注意力-梯度分数聚合为沿问题-思考-总结路径的步间映射。在多个模型中，Step-Saliency 揭示出两类重复出现的信息流失败：浅层锁定（Shallow Lock-in），即浅层过度聚焦当前步骤，几乎不利用早期上下文；深层衰减（Deep Decay），即深层逐渐丧失对思考段的显著性，总结部分越来越关注自身及最后几步。基于这些模式，提出 StepFlow，一种基于显著性的测试时干预：通过 Odds-Equal Bridge 调整浅层显著性模式，并通过 Step Momentum Injection 在深层添加小步级残差。StepFlow 在不重新训练的情况下，提升了多个 LRMs 在数学、科学和编程任务上的准确性，表明修复信息流可恢复部分缺失的推理能力。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) that generate long chains of thought now perform well on multi-step math, science, and coding tasks. However, their behavior is still unstable and hard to interpret, and existing analysis tools struggle with such long, structured reasoning traces. We introduce Step-Saliency, which pools attention--gradient scores into step-to-step maps along the question--thinking--summary trajectory. Across several models, Step-Saliency reveals two recurring information-flow failures: Shallow Lock-in, where shallow layers over-focus on the current step and barely use earlier context, and Deep Decay, where deep layers gradually lose saliency on the thinking segment and the summary increasingly attends to itself and the last few steps. Motivated by these patterns, we propose StepFlow, a saliency-inspired test-time intervention that adjusts shallow saliency patterns measured by Step-Saliency via Odds-Equal Bridge and adds a small step-level residual in deep layers via Step Momentum Injection. StepFlow improves accuracy on math, science, and coding tasks across multiple LRMs without retraining, indicating that repairing information flow can recover part of their missing reasoning performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。