首步错误会严重拖累后续推理，新方法仅保留优质首步可省70%算力。
Lost at the Beginning of Reasoning
- 通过奖励模型筛选高质量首步推理，避免无效冗余
- 首步错误导致整体推理质量下降，跨模型现象普遍
- 适合追求高效推理的部署场景，尤其对算力敏感任务
大型语言模型在复杂推理方面取得显著进展，尤其体现在包含回溯、自省和自修正机制的长链思维（CoT）推理上。然而，模型在长链推理中的自修正能力仍待深入探索。近期研究发现，模型常出现过度思考现象，即进行不必要的冗余推理。本文实证表明，首个推理步骤对最终预测具有不成比例的决定性影响：该阶段引入的错误会显著降低后续推理质量。这一现象在多个前沿开源与闭源推理模型中均被观察到。基于此，我们提出一种高效的采样策略，利用奖励模型识别并保留高质量首步推理，舍弃低质首步，实现高达70%的推理成本降低，且不损失准确率。本工作强调了首步推理在生成优质推理轨迹中的核心作用，从而支持更高效的采样。
原文摘要 · Abstract (English)
Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that incorporates mechanisms such as backtracking, self-reflection, and self-correction. Despite these developments, the self-correction abilities of LLMs during long CoT reasoning remain underexplored. And recent findings on overthinking suggest that such models often engage in unnecessarily redundant reasoning. In this work, we empirically show that the first reasoning step exerts a disproportionately large influence on the final prediction. I.e., errors introduced at this stage can substantially degrade subsequent reasoning quality. This phenomenon is consistently observed across various state-of-the-art open- and closed-source reasoning models. Leveraging this insight, we propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones, achieving up to a 70% reduction in inference cost without sacrificing any accuracy. Our work highlights the central role of the first reasoning step in generating a high-quality reasoning trajectory, and thus enabling significantly efficient sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。