arXiv:2603.27416cs.ROcs.AI2026-03被引 1

智能体自主迭代优化四足机器人行走策略,人类仅提供方向指导。

Agent-Driven Autonomous Reinforcement Learning Research: Iterative Policy Improvement for Quadruped Locomotion

  • 智能体自主完成代码读取、故障诊断、实验配置与执行全流程
  • 最终实现速度误差0.263、97%超时率的稳定行走,重复五次验证成功
  • 适合关注机器人强化学习自动化研究的工程师与科研人员

本文记录了一项关于四足机器人行走的智能体驱动自主强化学习研究的案例。人类通过代理编程环境提供高层指令,而智能体负责大部分执行循环:读取代码、诊断失败、修改奖励函数与地形配置、启动并监控实验、分析中间指标,并提出下一波实验方案。在Isaac Lab环境中对DHAV1 12-DoF四足机器人进行超过70次实验,分14个波次,智能体从初期平均奖励约7的粗糙地形行走,进步到第12波最佳实验exp063,速度误差0.263,2000次迭代中97%未超时,且在不同GPU上独立复现五次。研究还记录了多个自主决策实例:定位PhysX死锁源于含盒子和阶梯类原型的地形集,移植四个公开参考实现中的奖励项,修复Isaac Sim导入与启动问题,减少环境数量以加速诊断,终止卡死任务,以及在多次地形=0.0结果后放弃对HIM的投入。相较于AutoResearch范式,本案例在更易出错的机器人强化学习场景下,涉及多GPU实验管理与模拟器特定工程约束,贡献为实证与记录性:证明智能体可在有限人类干预下有效执行该领域的迭代强化学习流程,同时清晰揭示人类仍主导研究议程的关键节点。

原文摘要 · Abstract (English)

This paper documents a case study in agent-driven autonomous reinforcement learning research for quadruped locomotion. The setting was not a fully self-starting research system. A human provided high-level directives through an agentic coding environment, while an agent carried out most of the execution loop: reading code, diagnosing failures, editing reward and terrain configurations, launching and monitoring jobs, analyzing intermediate metrics, and proposing the next wave of experiments. Across more than 70 experiments organized into fourteen waves on a DHAV1 12-DoF quadruped in Isaac Lab, the agent progressed from early rough-terrain runs with mean reward around 7 to a best logged Wave 12 run, exp063, with velocity error 0.263 and 97\% timeout over 2000 iterations, independently reproduced five times across different GPUs. The archive also records several concrete autonomous research decisions: isolating PhysX deadlocks to terrain sets containing boxes and stair-like primitives, porting four reward terms from openly available reference implementations \cite{deeprobotics, rlsar}, correcting Isaac Sim import and bootstrapping issues, reducing environment count for diagnosis, terminating hung runs, and pivoting effort away from HIM after repeated terrain=0.0 outcomes. Relative to the AutoResearch paradigm \cite{autoresearch}, this case study operates in a more failure-prone robotics RL setting with multi-GPU experiment management and simulator-specific engineering constraints. The contribution is empirical and documentary: it shows that an agent can materially execute the iterative RL research loop in this domain with limited human intervention, while also making clear where human direction still shaped the agenda.

强化学习四足机器人自主研究智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。