arXiv:2505.10022cs.RO2025-05被引 1

让四足机器人学会自然运动,无需依赖演示动作即可自适应调整。

APEX: Action Priors Enable Efficient Exploration for Robust Motion Tracking on Legged Robots

  • 用逐渐衰减的动作先验引导探索,早期模仿演示动作,后期纯靠强化学习。
  • 在仿真中实现多地形、多速度下多样动作迁移,且对训练参数变化鲁棒。
  • 首次实现零样本部署于真实机器人,显著减少调参和对演示数据的依赖。

从示范中学习自然、类动物的运动已成为足式机器人领域的核心范式。尽管运动跟踪可复现参考步态,但许多方法仍需大量调参,并依赖部署时的参考运动输入,限制了对任务目标的响应性和适应性。我们提出 APEX(Action Priors enable Efficient eXploration),一种去除部署时对参考运动依赖、提升样本效率并减少调参的工作。APEX 通过衰减的动作先验将示范融入强化学习:初期引导探索向示范一致的动作靠近,随后渐弱至零,最终生成纯强化学习策略。同时结合多评判器框架,分离风格与任务+正则化学习信号。此外,单一策略可学习多种运动并跨地形、速度转移参考风格,且对训练参数变化保持鲁棒。我们在人形和四足机器人仿真中验证方法,并在 Unitree Go2 机器人上实现零样本部署。网站与代码:https://marmotlab.github.io/APEX/。

原文摘要 · Abstract (English)

Learning natural, animal-like locomotion from demonstrations has become a core paradigm in legged robotics. While motion tracking can reproduce reference gaits, many approaches still require substantial tuning and depend on reference motion inputs at deployment, which can limit responsiveness to task objectives and reduce adaptability. We present APEX (Action Priors enable Efficient eXploration), a motion-tracking reinforcement learning (RL) framework that removes deployment-time dependence on reference motion inputs, improves sample efficiency, and reduces tuning effort. APEX integrates demonstrations into RL via decaying action priors, which guide early exploration toward demonstration-consistent actions and then fade to zero, yielding a pure RL policy at deployment. This is combined with a multi-critic framework that separates style and task + regularization learning signals. Moreover, APEX enables a single policy to learn diverse motions and transfer reference-like styles across different terrains and velocities, while remaining robust to variations in training parameters. We validate our method in simulation on both humanoid and quadruped robots, and with zero-shot deployment on a Unitree Go2 robot. Website and code: https://marmotlab.github.io/APEX/.

足式机器人强化学习运动跟踪零样本部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。