提出三种统一的推理方法,分支法在多数场景下显著提升模型准确率。
Recursive Agentic Reasoning
- 将迭代、分解、采样等方法抽象为递归操作:生长、剪枝、分支
- 分支法平均提升5.98个百分点,在14个设置中12次表现最佳
- 揭示了分支法能有效应对输出截断问题,适合评估测试时计算的模型
测试时推理方法如迭代精炼、分解和重复采样常被孤立评估,难以跨模型、基准和评测流程比较。我们提出一种统一视角,将这些方法视为对智能体推理轨迹的递归操作:GROW(深化单一推理路径)、PRUNE(分解并重组问题)和BRANCH(采样多条推理路径并选择最优)。我们在相同提示、令牌预算和评分代码的统一框架下,对比三类操作与单步思维链基线。在五个基准和三个前沿模型共14个模型-基准组合中,涵盖49,327项评分样本和151,876次模型调用,BRANCH在所有14个设置中均提升准确率,平均增益5.98个百分点,且在12个设置中表现最佳;而GROW平均增益2.18点,但在两个设置中性能下降;PRUNE平均提升0.94点。分析表明,BRANCH的优势不仅来自多路径探索,更在于恢复因截断导致的失败输出:其增益与基线空输出率高度相关(r = 0.72)。结果削弱了‘不同问题需切换推理策略’的假设,在此抽象层次上,反复分支始终占优。最后,我们发现非配对评估及将评分管道失败误判为模型错误,会显著改变甚至反转比较结论,因此倡导配对评分作为测试时计算评估的标准协议。
原文摘要 · Abstract (English)
Test-time reasoning methods such as iterative refinement, decomposition, and repeated sampling are often evaluated in isolation, making their gains difficult to compare across models, benchmarks, and evaluation pipelines. We introduce a unified view of these methods as recursion operators over an agent's reasoning trace: GROW, which deepens a single reasoning path; PRUNE, which decomposes and recomposes the problem; and BRANCH, which samples alternative reasoning paths and selects among them. We evaluate all three operators against a single-pass chain-of-thought baseline under a shared harness with identical prompts, token budgets, and grading code. Across five benchmarks and three frontier models, comprising 14 model-benchmark settings, 49,327 graded items, and 151,876 model calls, BRANCH improves accuracy in all 14 settings by an average of 5.98 percentage points and is the best-performing operator in 12. In contrast, GROW yields a mean gain of 2.18 points and degrades performance in two settings, while PRUNE improves accuracy by 0.94 points on average. Analysis shows that BRANCH's advantage arises not only from exploring multiple reasoning paths, but also from recovering from truncation: its gains strongly correlate with the baseline rate of empty, budget-exhausted outputs (r = 0.72). These results weaken the hypothesis that different problems require routing among test-time reasoning operators; at this level of abstraction, repeated branching is consistently dominant. Finally, we show that unpaired evaluation and treating scoring-pipeline failures as model errors can materially change, and even reverse, comparative conclusions, motivating paired scoring as a standard protocol for test-time-compute evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。