让AI自主做机器学习工程,关键在动态更新知识而非死磕方案
Beyond Solution-Centric Search: Adaptive Inquiry and Knowledge Revision for Autonomous ML Engineering

- 用可修订的知识状态替代固定搜索路径,动态引导决策
- 12小时内达成64.9%获奖率,优于现有系统
- 适合需要跨领域迭代的复杂自动化任务
长周期自主研究任务如机器学习工程要求系统在有限预算下做出相互依赖的决策。现有基于大模型的智能体通常通过树、图或链结构组织候选方案改进,这种设计称为以解决方案为中心的搜索。本文提出信息范式,用不断演化的信息状态来表征系统对任务的理解,并指导方案优化。我们实现了名为Iris的探究-修正循环:在信息获取方面,从当前信息状态生成局部行动规划,利用认知性操作探查关键未知项而不修改已有方案;在信息管理方面,将实验观察整合为包含明确范围与状态的可修订命题构成的任务知识,并随新证据持续更新。每个决策上下文由原始证据、结构化摘要或所需粒度的任务知识构建。在MLE-Bench上,Iris在12小时预算下获得64.9%的任意奖牌率,是对比系统中最高的。在涵盖硬件工程与模型后训练的四项任务中,Iris也展现出跨领域泛化能力。
原文摘要 · Abstract (English)
Long-horizon autonomous research tasks such as machine learning engineering require systems to make interdependent decisions under a limited budget. Existing LLM-based agents typically organize candidate-solution improvement through tree, graph, or chain structures, meaning that the search process determines how information is acquired and managed. We call this design solution-centric search and propose instead the information paradigm, in which an evolving information state represents the system's understanding of the task and guides solution improvement. We instantiate this paradigm in Iris, an inquiry-revision loop. For information acquisition, Iris generates local action plans from the current information state and uses epistemic actions to probe decision-critical unknowns without modifying the retained solution. For information management, Iris synthesizes observations across experiments into task knowledge composed of revisable claims with explicit scope and status. It updates this knowledge as new evidence arrives and constructs each decision context from raw evidence, structured summaries, or task knowledge at the required level of detail. On MLE-Bench, Iris attains a 64.9% any-medal rate under a 12-hour budget, the highest among compared systems. Across four tasks spanning harness engineering and model post-training, Iris also demonstrates cross-domain generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。