arXiv:2601.16175cs.LGcs.AI2026-01被引 78

让AI在测试时持续学习,一次找到最优解。

Learning to Discover at Test Time

  • 测试时用强化学习让大模型持续训练,针对具体问题优化
  • 在数学、算法、生物等领域多项任务刷新纪录,最快达2倍提升
  • 仅需数百美元成本,开源模型和代码可复现

如何利用AI发现科学问题的新最优解?以往的测试时扩展方法(如AlphaEvolve)通过提示冻结的大语言模型进行搜索。本文提出测试时训练发现(TTT-Discover),在测试阶段进行强化学习,使大模型能持续训练,且学习经验针对具体测试问题。这种持续学习的目标是产生一个极优解,而非平均表现良好的多个解,也非泛化到其他问题。因此,其学习目标与搜索子程序均优先关注最有希望的解。我们在连续奖励的问题上评估该方法,涵盖数学、GPU内核工程、算法设计与生物学领域。所有尝试的问题均取得新最优结果:(i)解决Erdős最小重叠问题与自相关不等式;(ii)GPU内核竞赛中速度提升最高达2倍;(iii)复现并超越过往AtCoder算法比赛成绩;(iv)单细胞数据分析中的去噪问题。解决方案经专家或组织方评审。所有结果均使用开源模型OpenAI gpt-oss-120b,代码公开可复现,不同于以往依赖闭源前沿模型的结果。测试时训练运行基于Thinking Machines的Tinker API,每问题成本仅数百美元。

原文摘要 · Abstract (English)

How can we use AI to discover a new state of the art for a scientific problem? Prior work in test-time scaling, such as AlphaEvolve, performs search by prompting a frozen LLM. We perform reinforcement learning at test time, so the LLM can continue to train, but now with experience specific to the test problem. This form of continual learning is quite special, because its goal is to produce one great solution rather than many good ones on average, and to solve this very problem rather than generalize to other problems. Therefore, our learning objective and search subroutine are designed to prioritize the most promising solutions. We call this method Test-Time Training to Discover (TTT-Discover). Following prior work, we focus on problems with continuous rewards. We report results for every problem we attempted, across mathematics, GPU kernel engineering, algorithm design, and biology. TTT-Discover sets the new state of the art in almost all of them: (i) Erdős' minimum overlap problem and an autocorrelation inequality; (ii) a GPUMode kernel competition (up to $2\times$ faster than prior art); (iii) past AtCoder algorithm competitions; and (iv) denoising problem in single-cell analysis. Our solutions are reviewed by experts or the organizers. All our results are achieved with an open model, OpenAI gpt-oss-120b, and can be reproduced with our publicly available code, in contrast to previous best results that required closed frontier models. Our test-time training runs are performed using Tinker, an API by Thinking Machines, with a cost of only a few hundred dollars per problem.

测试时学习强化学习科学发现大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。