用特权经验训练四足机器人，实现视觉导航下的敏捷安全运动
SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience
- 先用特权信息训练初始策略，再用其生成数据来指导视觉强化学习
- 在真实Solo-12机器人上实现跳跃、攀爬、爬行等多种跑酷动作
- 避免直接从像素训练的高成本，提升样本效率与安全性
跑酷对四足机器人构成重大挑战，需在有限感知下实现敏捷精准的环境穿越。本文提出一种端到端视觉策略训练新方法，从深度图像直接输出机器人控制指令，实现敏捷且安全的四足运动。将机器人跑酷建模为约束强化学习问题，在最大化技能涌现的同时确保符合物理极限。首先利用关于环境的特权信息训练无视觉的策略；随后基于该策略生成经验，用于启动高效的离线策略强化学习算法，仅从深度图像进行训练。此方法使机器人能从特权经验中迁移行为，规避直接从像素训练的高计算开销。我们在真实Solo-12机器人上验证了该方法的有效性，展示了其完成行走、攀爬、跳跃和爬行等多种跑酷技能的能力。
原文摘要 · Abstract (English)
Parkour poses a significant challenge for legged robots, requiring navigation through complex environments with agility and precision based on limited sensory inputs. In this work, we introduce a novel method for training end-to-end visual policies, from depth pixels to robot control commands, to achieve agile and safe quadruped locomotion. We formulate robot parkour as a constrained reinforcement learning (RL) problem designed to maximize the emergence of agile skills within the robot's physical limits while ensuring safety. We first train a policy without vision using privileged information about the robot's surroundings. We then generate experience from this privileged policy to warm-start a sample efficient off-policy RL algorithm from depth images. This allows the robot to adapt behaviors from this privileged experience to visual locomotion while circumventing the high computational costs of RL directly from pixels. We demonstrate the effectiveness of our method on a real Solo-12 robot, showcasing its capability to perform a variety of parkour skills such as walking, climbing, leaping, and crawling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。