用推理模型区分假说与实验问题,提升自动化科学发现效率
Learning the ARTS of Search for Automated Discovery

- 用语言模型分析历史实验,判断失败是假说错还是执行差
- 在22个任务上比现有方法高15.3%的标准化得分
- 低成本模型经测试时训练后可媲美顶级闭源模型
科学发现可建模为在假设与实验空间中的迭代搜索过程。当前方法多使用如MCTS等启发式算法,但会混淆假说价值与实验执行质量,导致初步表现不佳的优秀假说被低估。此外,随着搜索进展,历史记录超出上下文窗口,现有方法需裁剪日志。本文提出代理推理树搜索(ARTS),部署推理型语言模型来导航该空间。模型分析过往执行记录,诊断早期失败是源于错误实现还是劣质假说,并选择下一阶段应改进的假说。为缓解上下文长度限制,ARTS采用测试时训练,将搜索树知识注入模型权重。在MLGym和MLEBench的22项任务中,ARTS性能显著优于主流算法,标准化得分提升超15.3%。测试时训练下,Qwen3-4B代理可达到与Gemini-3 Pro和GPT o3-reasoning相当的性能,推理成本降低最多达5倍。进一步发现在部分可观测强化学习任务中,经测试时训练的Qwen3-4B科学家甚至重发现了人类最优的循环记忆解法,而启发式方法因剪枝将其遗漏。
原文摘要 · Abstract (English)
Scientific discovery can be formulated as an iterative search process over the space of hypotheses and experiments. Contemporary methods navigate this space using heuristics such as MCTS. These algorithms conflate the merit of a hypothesis with the quality of its experimental execution. A promising hypothesis with preliminary execution is therefore ranked below a modest hypothesis whose execution is refined. Moreover, prior methods prune the search logs as the search progresses because the accumulated history outgrows the context window. We propose Agentic Reasoning for Tree Search (ARTS), where we deploy a reasoning language model to navigate this space. The model inspects prior execution logs, diagnoses whether earlier failures arose from faulty implementations or bad hypotheses, and selects the hypothesis to build on next. To mitigate challenges with context length, ARTS uses test-time training to instill the knowledge of search tree in the model weights. Across 22 tasks from MLGym and MLEBench, we show that ARTS outperforms leading algorithms, with over 15.3% relative improvement in the normalized score. With test-time training we show that a Qwen3-4B agent can match performance with closed-source frontier models like Gemini-3 Pro and GPT o3-reasoning with upto 5x lower inference cost. We further observe that on partially observable RL tasks, the test-time trained Qwen3-4B scientist surpasses ARTS with the o3 scientist by rediscovering the human-best recurrent-memory solution that heuristic methods prune away.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。