arXiv:2503.01255cs.RO2025-03被引 9

忽略静摩擦会拉大仿真到现实的差距,新方法有效提升机器人落地表现。

Impact of Static Friction on Sim2Real in Robotic Reinforcement Learning

  • 引入静摩擦感知的领域随机化,改进仿真训练策略。
  • 在楼梯等复杂地形上,新方法比传统方法提升显著性能。
  • 适合关注机器人真实场景部署的研究者与工程师。

在机器人强化学习中,仿真到现实(Sim2Real)的差距仍是关键挑战。然而,静摩擦对这一差距的影响尚未充分研究。传统领域随机化方法通常不将静摩擦纳入参数空间,导致真实世界模型性能大幅下降。为此,我们采用执行器网络替代传统方法,在平坦地面实现成功迁移,但在楼梯等复杂地形失败。通过构建控制理论级关节模型并系统识别参数,发现机器人关节存在异常高的摩擦力矩比。为缓解此问题,提出静摩擦感知的领域随机化方法,并设计简化学习复杂度的新策略。通过对比三种方法(传统领域随机化、执行器网络、本方法)在模拟-模拟和模拟-现实环境中的实验,均使用快速运动适应(RMA)算法,结果表明本方法在自适应能力与整体性能上表现最优。

原文摘要 · Abstract (English)

In robotic reinforcement learning, the Sim2Real gap remains a critical challenge. However, the impact of Static friction on Sim2Real has been underexplored. Conventional domain randomization methods typically exclude Static friction from their parameter space. In our robotic reinforcement learning task, such conventional domain randomization approaches resulted in significantly underperforming real-world models. To address this Sim2Real challenge, we employed Actuator Net as an alternative to conventional domain randomization. While this method enabled successful transfer to flat-ground locomotion, it failed on complex terrains like stairs. To further investigate physical parameters affecting Sim2Real in robotic joints, we developed a control-theoretic joint model and performed systematic parameter identification. Our analysis revealed unexpectedly high friction-torque ratios in our robotic joints. To mitigate its impact, we implemented Static friction-aware domain randomization for Sim2Real. Recognizing the increased training difficulty introduced by friction modeling, we proposed a simple and novel solution to reduce learning complexity. To validate this approach, we conducted comprehensive Sim2Sim and Sim2Real experiments comparing three methods: conventional domain randomization (without Static friction), Actuator Net, and our Static friction-aware domain randomization. All experiments utilized the Rapid Motor Adaptation (RMA) algorithm. Results demonstrated that our method achieved superior adaptive capabilities and overall performance.

Sim2Real强化学习机器人控制静摩擦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。