让大模型在测试时自我迭代,无需外部训练即可提升推理能力。
Test-time Recursive Thinking: Self-Improvement without External Feedback
- 通过滚动策略与自验证信号,动态优化生成过程
- 开源模型在AIME-25/24上达100%准确率
- 闭源模型在复杂编码任务上提升10.4%-14.8%
现代大语言模型的推理能力迅速提升,主要依赖于可验证奖励的强化学习。本文探讨这些模型是否能在无需额外训练的情况下实现自我改进。我们识别出两大挑战:(i) 高效生成多样且高质量的候选解;(ii) 在缺乏真实标签监督的情况下可靠选择正确答案。为此,提出测试时递归思考(TRT)框架,该框架基于滚动特定策略、累积知识和自生成验证信号进行生成。使用TRT后,开源模型在AIME-25/24数据集上达到100%准确率;在LiveCodeBench最困难的问题上,闭源模型性能提升10.4-14.8个百分点,且无需外部反馈。
原文摘要 · Abstract (English)
Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whether these LLMs can self-improve without the need for additional training. We identify two core challenges for such systems: (i) efficiently generating diverse, high-quality candidate solutions, and (ii) reliably selecting correct answers in the absence of ground-truth supervision. To address these challenges, we propose Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals. Using TRT, open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。