arXiv:2503.08349cs.RO2025-03被引 7

提出LiPS方法,让类人机器人在仿真中直接训练闭环结构。

LiPS: Large-Scale Humanoid Robot Reinforcement Learning with Parallel-Series Structures

  • 在仿真中引入多刚体动力学建模,支持闭环结构训练
  • 实现类人机器人大规模并行强化学习,减少真实世界部署难度
  • 适合研究复杂机械结构机器人控制的团队

近年来,类人机器人研究受到广泛关注,基于强化学习的控制算法取得了显著突破。相比传统模型驱动方法,强化学习在处理复杂任务时更具优势。借助GPU的大规模并行计算能力,现代类人机器人可在仿真环境中进行广泛并行训练。然而,当前多数强化学习控制算法在训练阶段采用开环拓扑,将串并联结构转换推迟到sim2real阶段,主要受限于物理引擎对多刚体闭合回路模拟的支持不足。为此,我们提出LiPS方法,通过在仿真环境中引入多刚体动力学建模,显著缩小sim2real差距,并降低模型部署时向并联结构转换的难度,从而有效支撑类人机器人的大规模强化学习训练。

原文摘要 · Abstract (English)

In recent years, research on humanoid robots has garnered significant attention, particularly in reinforcement learning based control algorithms, which have achieved major breakthroughs. Compared to traditional model-based control algorithms, reinforcement learning based algorithms demonstrate substantial advantages in handling complex tasks. Leveraging the large-scale parallel computing capabilities of GPUs, contemporary humanoid robots can undergo extensive parallel training in simulated environments. A physical simulation platform capable of large-scale parallel training is crucial for the development of humanoid robots. As one of the most complex robot forms, humanoid robots typically possess intricate mechanical structures, encompassing numerous series and parallel mechanisms. However, many reinforcement learning based humanoid robot control algorithms currently employ open-loop topologies during training, deferring the conversion to series-parallel structures until the sim2real phase. This approach is primarily due to the limitations of physics engines, as current GPU-based physics engines often only support open-loop topologies or have limited capabilities in simulating multi-rigid-body closed-loop topologies. For enabling reinforcement learning-based humanoid robot control algorithms to train in large-scale parallel environments, we propose a novel training method LiPS. By incorporating multi-rigid-body dynamics modeling in the simulation environment, we significantly reduce the sim2real gap and the difficulty of converting to parallel structures during model deployment, thereby robustly supporting large-scale reinforcement learning for humanoid robots.

类人机器人强化学习仿真训练多刚体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。