提出自适应课程强化学习框架，提升自动机器学习的持续优化能力
AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineering
- 构建可迭代更新的数据缓冲区，复用执行轨迹生成训练任务
- 通过可学习潜力函数动态选择高价值任务，提升学习效率
- 在小型基准上实现100%有效提交率，超越更大开源模型
自主机器学习工程（MLE）要求智能体在长时序中持续进行迭代优化。尽管基于大模型的智能体展现潜力，但当前基于提示的代理因参数冻结导致行为停滞。虽强化学习可缓解此问题，但在MLE中受限于高昂的执行延迟和低效的数据选择。为此，我们提出AceGRPO，包含两个核心组件：(1) 可演化数据缓冲区，持续将执行轨迹转化为可复用的训练任务；(2) 基于可学习潜力函数的自适应采样，动态优先选择处于智能体学习前沿的任务，以最大化学习效率。基于AceGRPO训练的Ace-30B模型在MLE-Bench-Lite上实现100%有效提交率，逼近专有前沿模型性能，并优于更大的开源基线（如DeepSeek-V3.2），展现出强大的持续迭代优化能力。代码已公开于https://github.com/yuzhu-cai/AceGRPO。
原文摘要 · Abstract (English)
Autonomous Machine Learning Engineering (MLE) requires agents to perform sustained, iterative optimization over long horizons. While recent LLM-based agents show promise, current prompt-based agents for MLE suffer from behavioral stagnation due to frozen parameters. Although Reinforcement Learning (RL) offers a remedy, applying it to MLE is hindered by prohibitive execution latency and inefficient data selection. Recognizing these challenges, we propose AceGRPO with two core components: (1) Evolving Data Buffer that continuously repurposes execution traces into reusable training tasks, and (2) Adaptive Sampling guided by a Learnability Potential function, which dynamically prioritizes tasks at the agent's learning frontier to maximize learning efficiency. Leveraging AceGRPO, our trained Ace-30B model achieves a 100% valid submission rate on MLE-Bench-Lite, approaches the performance of proprietary frontier models, and outperforms larger open-source baselines (e.g., DeepSeek-V3.2), demonstrating robust capability for sustained iterative optimization. Code is available at https://github.com/yuzhu-cai/AceGRPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。