发现有效推理的关键是减少失败步骤，而非越长越好。
What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
- 用图结构分析推理路径，提取失败步骤比例作为核心指标。
- 失败步骤比例越高，准确率越低，优于长度和重访次数等传统指标。
- 删除失败分支可显著提效，适合追求推理质量的模型优化者。
大型推理模型在测试阶段消耗大量计算资源生成长链式思维（CoT），但有效CoT的特征尚不明确。尽管先前研究认为延长CoT和增加回顾（通过附加等待标记实现）能提升性能，但近期研究表明更短的思考过程反而表现更优。为此，我们对十种大型推理模型在数学与科学推理任务上进行了系统评估。结果表明，盲目延长CoT或增加回顾次数均与更低的准确率相关。随着推理逐步展开，逐标记指标容易将冗余表达与真实推理质量混淆。为此，我们提出一种CoT的图结构视图，从中提取关键统计量——失败步骤比例（FSF），即被放弃分支中的步骤占比。该指标在不同模型间均能更准确预测正确性。为进一步验证因果关系，我们设计两种干预：一是在测试时按各指标排序候选CoT，FSF带来的pass@1提升最大；二是在推理后编辑移除失败分支，显著提高准确率，表明失败分支会误导后续推理。综合来看，有效推理应以‘少失败’为特征，并支持基于结构感知的测试时扩展，而非无差别生成长推理链。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) spend substantial test-time compute on long chain-of-thought (CoT) traces, but what *characterizes* an effective CoT remains unclear. While prior work reports gains from lengthening CoTs and increasing review (revisiting earlier steps) via appended *wait* tokens, recent studies suggest that shorter thinking can outperform longer traces. We therefore conduct a systematic evaluation across ten LRMs on math and scientific reasoning. Contrary to the "longer-is-better" narrative, we find that both naive CoT lengthening and increased review are associated with *lower* accuracy. As CoT unfolds step by step, token-level metrics can conflate verbosity with process quality. We introduce a graph view of CoT to extract structure and identify a single statistic-the *Failed-Step Fraction (FSF)*, the fraction of steps in abandoned branches-that consistently outpredicts length and review ratio for correctness across models. To probe causality, we design two interventions. First, we rank candidate CoTs by each metric at test time, where FSF yields the largest pass@1 gains; second, we edit CoTs to remove failed branches, which significantly improves accuracy, indicating that failed branches bias subsequent reasoning. Taken together, these results characterize effective CoTs as those that *fail less* and support *structure-aware* test-time scaling over indiscriminately generating long CoT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。