让AI自动发现新机器学习算法,还能持续进化。
MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery

- 用图结构实现多分支信息共享,逐步从广搜转为精搜。
- 在12小时任务中达成最高有效提交率和奖牌率。
- 适合需要长期优化的自动化算法研发场景。
大型语言模型(LLM)代理在科学发现和机器学习工程(MLE)等长周期任务中应用日益广泛,持续自我进化成为关键能力。然而现有MLE代理存在跨分支信息隔离、无记忆搜索和缺乏层级控制等问题,阻碍了长周期优化。我们提出MLEvolve,一个基于LLM的自进化多代理框架,用于端到端机器学习算法发现。通过将树搜索扩展为渐进式MCGS,MLEvolve利用基于图的引用边实现跨分支信息流动,并通过熵启发的渐进调度,逐步将搜索从广度探索转向聚焦利用。为使代理能基于积累经验持续进化,我们引入回顾性记忆,结合冷启动领域知识库与动态全局记忆,实现任务特定经验的检索与复用。为保障长周期迭代的稳定性,进一步将策略规划与代码生成解耦,采用自适应编码模式。在MLE-Bench上的评估显示,MLEvolve在12小时预算下(为标准运行时间的一半),平均奖牌率和有效提交率均达到当前最优。此外,其在数学算法优化任务上也优于专用算法发现方法AlphaEvolve,展现出强大的跨领域泛化能力。代码已开源:https://github.com/InternScience/MLEvolve。
原文摘要 · Abstract (English)
Large language model (LLM) agents are increasingly applied to long-horizon tasks such as scientific discovery and machine learning engineering (MLE), where sustained self-evolution becomes a key capability. However, existing MLE agents suffer from inter-branch information isolation, memoryless search, and lack of hierarchical control, which together hinder long-horizon optimization. We present MLEvolve, an LLM-based self-evolving multi-agent framework for end-to-end machine learning algorithm discovery. By extending tree search to Progressive MCGS, MLEvolve enables cross-branch information flow through graph-based reference edges and gradually shifts the search from broad exploration to focused exploitation with an entropy-inspired progressive schedule. To allow the agent to evolve with accumulated experience, we introduce Retrospective Memory, which combines a cold-start domain knowledge base with a dynamic global memory for task-specific experience retrieval and reuse. For stable long-horizon iteration, we further decouple strategic planning from code generation with adaptive coding modes. Evaluation on MLE-Bench shows that MLEvolve achieves state-of-the-art performance across multiple dimensions including average medal rate and valid submission rate under a 12-hour budget (half the standard runtime). Moreover, MLEvolve also outperforms specialized algorithm discovery methods including AlphaEvolve on mathematical algorithm optimization tasks, demonstrating strong cross-domain generalization. Our code is available at https://github.com/InternScience/MLEvolve.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。