发现大模型推理错误多源于早期关键节点，可提前干预纠正。
Dissecting Failure Dynamics in Large Language Model Reasoning
- 通过分析推理路径，定位早期错误转折点
- 错误后推理仍局部连贯，但全局偏离正确方向
- 提出基于不确定性的干预框架，提升推理可靠性
大型语言模型通过延长推理过程获得优异表现，但其推理失败的机制仍不清晰。通过对模型生成的推理轨迹进行分析，我们发现错误并非均匀分布，而是常始于少数早期转换节点，此后推理虽保持局部连贯，却走向全局错误。这些节点对应令牌级熵的局部突增，且从同一中间状态出发的其他延续路径仍可能导向正确答案。基于此，我们提出GUARD——一种基于不确定性信号探测并重定向关键转换的推理时干预框架。在多个基准上的实证评估表明，由失败动态引导的干预能显著提升推理可靠性。研究强调了理解推理何时何地首次偏离的重要性，补充了现有侧重于扩展推理计算量的方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains poorly understood. By analyzing model-generated reasoning trajectories, we find that errors are not uniformly distributed but often originate from a small number of early transition points, after which reasoning remains locally coherent but globally incorrect. These transitions coincide with localized spikes in token-level entropy, and alternative continuations from the same intermediate state can still lead to correct solutions. Based on these observations, we introduce GUARD, a targeted inference-time framework that probes and redirects critical transitions using uncertainty signals. Empirical evaluations across multiple benchmarks confirm that interventions guided by these failure dynamics lead to more reliable reasoning outcomes. Our findings highlight the importance of understanding when and how reasoning first deviates, complementing existing approaches that focus on scaling inference-time computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。