测试大模型能否自主完成强化学习后训练全流程，发现部分可实现但稳定性不足。
Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?
- 构建交互式评估框架，让代理在固定预算内自主设计并执行训练流程。
- 某实验将ALFWorld任务表现从4.85提升至93.28，但多数任务仍依赖监督训练。
- 适用于研究智能体自主优化模型的学者，尤其关注强化学习与自动化训练者。
我们提出Agent2 RL-Bench，一个用于评估大模型智能体在强化学习后训练中自主能力的诊断性基准。该基准测试智能体是否能独立完成训练流程的设计、实现、调试与执行以提升基础模型性能。强化学习后训练正日益推动模型对齐与专业化，但现有基准多为静态，仅奖励监督微调或脚本生成，未能评估智能体闭环强化学习的能力。Agent2 RL-Bench提供统一的智能体接口：每次运行始于一个隔离工作区，包含基础模型、任务数据、指令和评分接口，智能体需在固定预算内通过训练模型并提交成果进行评估。基准涵盖六个任务，分三个层级：从静态规则训练到基于判断的优化，再到包含轨迹收集的闭环在线强化学习。引入运行时记录与事后总结两项诊断技能，支持对智能体行为的结构化分析，促进评估框架的迭代优化。在五个智能体系统与六个驱动大模型上，智能体表现出一定智能，但存在明显局限：一例采用SFT预热与GRPO在线采样的实验使ALFWorld得分从4.85升至93.28；然而DeepSearchQA仍困难重重，多数成功路径依赖监督范式，且不同智能体堆栈间单次运行结果差异显著。总体而言，当前智能体虽偶能实现在线强化学习，但在固定预算下稳定驱动强化学习后训练仍罕见。该基准也证明了其作为未来研究有效评估框架的潜力。代码详见：https://github.com/microsoft/RD-Agent/blob/main/rdagent/scenarios/rl/autorl_bench/README.md。
原文摘要 · Abstract (English)
We introduce Agent2 RL-Bench, a compact diagnostic benchmark for evaluating agentic RL post-training, which tests whether LLM agents can autonomously design, implement, debug, and execute post-training pipelines that improve foundation models. RL post-training increasingly drives model alignment and specialization, yet existing benchmarks are largely static, rewarding supervised fine-tuning or script generation without assessing an agent's ability to close an interactive RL loop. Agent2 RL-Bench provides a unified agent-facing interface: each run starts from an isolated workspace containing a base model, task data, instructions, and a grading API, and agents must iterate within a fixed budget by training models and submitting artifacts for evaluation. The benchmark spans six tasks across three levels, from static rule-based training to judge-based optimization and closed-loop online RL with trajectory collection. Two diagnostic skills, namely runtime recording and post-hoc summarization, enable structured analysis of agent behavior, facilitating smooth and effective iteration of the benchmark's evaluation framework. Across five agent systems and six driver LLMs, agents show intelligent behavior but clear limitations: one RL-oriented run improves ALFWorld from 4.85 to 93.28 via SFT warm-up and GRPO with online rollouts, yet DeepSearchQA remains difficult, most successful routes rely on supervised pipelines, and interactive outcomes show large single-run differences across agent stacks. Overall, Agent2 RL-Bench shows that current agents can sometimes engineer online RL, but stable agent-driven RL post-training remains rare under fixed budgets. It also demonstrates that our benchmark provides a strong and effective evaluation framework for future research in this direction. Code is available at https://github.com/microsoft/RD-Agent/blob/main/rdagent/scenarios/rl/autorl_bench/README.md
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。