arXiv:2608.05643cs.AIcs.CL2026-08

用自修正提升大模型推理,让测试时计算更有效。

Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning

论文配图:Refining Over Resampling: Test-Time Self-Correction for LLM Reasoning
图 1 · 摘自论文原文
  • 多路采样后通过自我批评迭代修正答案,兼顾广度与深度。
  • 在MATH500上准确率提升至58.0%,AMC达32.5%。
  • 无需外部验证器,适合开放权重模型的推理优化。

测试时扩展通过额外推理计算提升大模型推理能力,但单纯扩大采样范围常因重复模式而收益递减。验证器引导的选择依赖外部奖励模型校准,存在局限性。本文提出一种无需验证器的广度-深度精炼框架,利用测试时计算同时探索和改进候选解。方法对多个独立推理路径进行采样,通过迭代自我批判与修正提升质量,并以多数投票聚合结果。广度保持初始尝试多样性,深度修复局部推理错误。在AIME24、AIME25、AMC、OlympiadBench和MATH500数据集上,该方法持续优于贪心解码、多数投票、验证器引导的best-of-N、束搜索和前瞻解码。例如,在Qwen2.5-1.5B模型上,MATH500准确率从最强基线提升至58.0%,AMC从25.0%提升至32.5%。结果表明,测试时计算用于精炼采样轨迹比单纯增加采样量或依赖验证器更高效。

原文摘要 · Abstract (English)

Test-time scaling improves LLM reasoning by using additional inference compute, but wider sampling alone can suffer from diminishing returns: new rollouts often repeat existing answer patterns instead of adding useful reasoning diversity. Verifier-based selection offers an alternative, but its performance depends on the calibration of an external reward model. We propose a verifier-free breadth--depth refinement framework that uses test-time compute to both explore and improve candidate solutions. The method samples multiple independent reasoning rollouts, refines each rollout through iterative self-critique and self-correction, and aggregates the refined answers by majority voting. Breadth preserves diverse initial attempts, while depth repairs local reasoning errors before aggregation. Across AIME24, AIME25, AMC, OlympiadBench, and MATH500, our method consistently improves over greedy decoding, majority voting, verifier-based best-of-$N$, beam search, and lookahead decoding across multiple open-weight models. For instance, with Qwen2.5-1.5B, accuracy increases from the strongest verifier-based baseline to $58.0\%$ on MATH500, and from $25.0\%$ to $32.5\%$ on AMC. These results show that test-time compute can be more effective when used to refine sampled trajectories rather than only to sample more candidates or rely on verifier-guided selection.

大模型推理自修正测试时扩展

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。