用分布距离优化仿真参数，让机器人跑步速度超原厂三倍
High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures
- 用Wasserstein和MMD衡量仿真与真实数据差异，指导参数优化
- 训练出可实现5.2ms步频、支持多种步态的高敏捷控制策略
- 代码开源，支持低层接口的未来机器人强化学习研究
本文介绍了在波士顿动力Spot机器人上部署高性能强化学习策略的技术细节，使用Spot RL研究人员开发套件实现底层电机访问。这是首个公开端到端强化学习策略在Spot硬件上的演示，训练代码通过Nvidia IsaacLab公开，部署代码由波士顿动力提供。我们采用Wasserstein距离和最大均值差异（MMD）量化仿真与真实数据的分布差异，评估模拟到现实的差距，并将其作为协方差矩阵自适应进化策略的评分函数，以优化无法直接测量的仿真参数。该方法生成的策略可实现多种步态，包括飞行相。部署的策略最高可达5.2毫秒步频，超过原厂控制器最大速度三倍以上，具备防滑表面鲁棒性、抗扰动能力和前所未有的整体敏捷性。
原文摘要 · Abstract (English)
This work presents an overview of the technical details behind a high performance reinforcement learning policy deployment with the Spot RL Researcher Development Kit for low level motor access on Boston Dynamics Spot. This represents the first public demonstration of an end to end end reinforcement learning policy deployed on Spot hardware with training code publicly available through Nvidia IsaacLab and deployment code available through Boston Dynamics. We utilize Wasserstein Distance and Maximum Mean Discrepancy to quantify the distributional dissimilarity of data collected on hardware and in simulation to measure our sim2real gap. We use these measures as a scoring function for the Covariance Matrix Adaptation Evolution Strategy to optimize simulated parameters that are unknown or difficult to measure from Spot. Our procedure for modeling and training produces high quality reinforcement learning policies capable of multiple gaits, including a flight phase. We deploy policies capable of over 5.2ms locomotion, more than triple Spots default controller maximum speed, robustness to slippery surfaces, disturbance rejection, and overall agility previously unseen on Spot. We detail our method and release our code to support future work on Spot with the low level API.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。