AI自动探索神经网络架构，无需人工干预即可持续优化性能。
AutoResearch-RL: Perpetual Self-Evaluating Reinforcement Learning Agents for Autonomous Neural Architecture Discovery
- 用强化学习自动修改代码并测试，每轮在固定时间内评估效果。
- 300次迭代后达到或超过人工调优的性能，单卡完成。
- 适合对自动化机器学习感兴趣的研究者和工程师。
我们提出AutoResearch-RL，一种无需人工干预的强化学习框架，可自主进行开放式神经网络架构与超参数研究，持续运行直至收敛或资源耗尽。每步中，智能体对目标训练脚本（train.py）提出代码修改，在固定时长内执行，基于验证集比特/字节（val-bpb）获得标量奖励，并通过近端策略优化（PPO）更新策略。核心设计将三个要素分离：(i) 冻结环境（数据管道、评估协议与常量），确保跨实验公平性；(ii) 可变目标文件（train.py），代表智能体可编辑状态；(iii) 元学习器（即强化学习智能体），积累实验轨迹并指导后续提案。我们将此建模为马尔可夫决策过程，在弱假设下推导收敛性保证，并在单张GPU的nanochat预训练基准上实证表明，经约300次夜间迭代后，该框架发现的配置性能匹配或超越人工调优基线。
原文摘要 · Abstract (English)
We present AutoResearch-RL, a framework in which a reinforcement learning agent conducts open-ended neural architecture and hyperparameter research without human supervision, running perpetually until a termination oracle signals convergence or resource exhaustion. At each step the agent proposes a code modification to a target training script, executes it under a fixed wall clock time budget, observes a scalar reward derived from validation bits-per-byte (val-bpb), and updates its policy via Proximal Policy Optimisation (PPO). The key design insight is the separation of three concerns: (i) a frozen environment (data pipeline, evaluation protocol, and constants) that guarantees fair cross-experiment comparison; (ii) a mutable target file (train.py) that represents the agent's editable state; and (iii) a meta-learner (the RL agent itself) that accumulates a growing trajectory of experiment outcomes and uses them to inform subsequent proposals. We formalise this as a Markov Decision Process, derive convergence guarantees under mild assumptions, and demonstrate empirically on a single GPU nanochat pretraining benchmark that AutoResearch-RL discovers configurations that match or exceed hand-tuned baselines after approximately 300 overnight iterations, with no human in the loop.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。