arXiv:2510.08511cs.AIcs.CL2025-10被引 16

AutoMLGen用图搜索提升编码智能体在机器学习工程中的优化能力

AutoMLGen: Navigating Fine-Grained Optimization for Coding Agents

  • 引入领域知识库与蒙特卡洛图搜索,实现路径动态重组和历史轨迹复用
  • 在12小时预算下,平均奖牌率和有效提交率均达当前最优水平
  • 适合需要高精度自动调参的机器学习竞赛与工程场景

大语言模型在通用编程任务中表现优异,但在机器学习工程(MLE)场景如自动机器学习和Kaggle竞赛中,高性能依赖专家干预与反复调试,单纯生成正确代码难以奏效。直接应用时,LLMs缺乏细粒度领域先验,现有基于线性或树结构的搜索方法仅能传递相邻层级知识,无法利用完整历史轨迹或跨分支信息共享,限制了自演化能力和搜索空间多样性。为此,我们提出AutoMLGen,一种基于大语言模型的编码智能体,融合领域知识库提供高质量先验引导,并采用蒙特卡洛图搜索(MCGS)实现高效探索。MCGS在保留树形引导探索的同时,将图结构嵌入扩展阶段,支持动态路径重组、历史轨迹重用与多解融合,促进自演化与协同学习。结合细粒度算子集设计,显著提升稳定性并加速收敛。在MLE-Bench上的评估显示,于12小时预算(为标准运行时间一半)下,AutoMLGen在平均奖牌率和有效提交率等多项指标上达到当前最优性能。代码已开源:https://github.com/Alpha-Innovator/InternAgent。

原文摘要 · Abstract (English)

Large language models (LLMs) have shown impressive performance in general programming tasks. However, in Machine Learning Engineering (MLE) scenarios such as AutoML and Kaggle competitions, achieving high performance depends heavily on expert intervention and repeated adjustments rather than simply generating correct code. When applied directly to these tasks, LLMs often lack fine-grained domain priors, and existing MLE approaches that use linear or tree-structured searches limit knowledge transfer to adjacent hierarchical links. As a result, they cannot leverage past full trajectories or share information across branches, limiting self-evolving ability and search space diversity. To address these limitations, we introduce AutoMLGen, an LLM-based coding agent that integrates a domain knowledge base for high-quality prior guidance and Monte Carlo Graph Search (MCGS) for efficient exploration. MCGS retains the tree-guided exploration of MCTS while embedding a graph structure into the expansion stage to enable dynamic path reorganization, historical trajectory reuse, and multi-solution fusion to support both self-evolution and collaborative learning. Combined with fine-grained operator sets, this design improves stability and accelerates convergence. Evaluation on the MLE-Bench shows that AutoMLGen achieves state-of-the-art performance in numerous dimensions, such as the average medal rate and the valid submission rate, under a 12-hour budget (half the standard runtime). The code is available at https://github.com/Alpha-Innovator/InternAgent.

AutoML代码生成强化学习智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。