用可泛化的力模型缩小仿真到现实的差距,让绳驱机械指更精准抓握。
Tendon Force Modeling for Sim2Real Transfer of Reinforcement Learning Policies for Tendon-Driven Robots
- 基于变压器模型预测电机产生的绳索受力,融合历史上下文信息。
- 在真实绳驱手指上实现50%的末端位姿追踪精度提升,仿真到现实误差降低41%。
- 方法不依赖特定机器人,适合各类绳驱系统和强化学习控制策略。
采用软体或柔顺交互的机器人常使用绳驱驱动,使执行器布局更灵活并保持柔顺性。然而,控制复杂绳索系统极具挑战。结合强化学习(RL)的仿真可生成更复杂行为,但现有方法依赖扭矩与力的仿真推演,受限于仿真到现实的差距,源于执行器与系统动力学差异,导致RL策略难以迁移至真实机器人。为此,本文提出一种建模典型伺服电机产生绳索力的方法,聚焦绳驱手指的策略迁移。该方法扩展了数据驱动技术,利用上下文历史和新型数据采集测试平台,捕捉真实操作中丰富的接触交互。随后,将力估计模型集成至GPU加速的绳索力驱动刚体仿真中训练基于强化学习的控制器。所提出的变压器模型能将绳索力预测误差控制在最大电机力的3%以内,且具备机器人无关性。通过引入学习模型,测试轨迹的仿真到现实差距减少41%。基于该模型训练的控制器在真实绳驱手指上实现了50%的末端位姿追踪任务性能提升。该方法可推广至不同执行器与机器人系统,推动灵巧操作器与软体机器人的广泛应用。
原文摘要 · Abstract (English)
Robots which make use of soft or compliant inter- actions often leverage tendon-driven actuation which enables actuators to be placed more flexibly, and compliance to be maintained. However, controlling complex tendon systems is challenging. Simulation paired with reinforcement learning (RL) could be enable more complex behaviors to be generated. Such methods rely on torque and force-based simulation roll- outs which are limited by the sim-to-real gap, stemming from the actuator and system dynamics, resulting in poor transfer of RL policies onto real robots. To address this, we propose a method to model the tendon forces produced by typical servo motors, focusing specifically on the transfer of RL policies for a tendon driven finger. Our approach extends existing data- driven techniques by leveraging contextual history and a novel data collection test-bench. This test-bench allows us to capture tendon forces undergo contact-rich interactions typical of real- world manipulation. We then utilize our force estimation model in a GPU-accelerated tendon force-driven rigid body simulation to train RL-based controllers. Our transformer-based model is capable of predicting tendon forces within 3% of the maximum motor force and is robot-agnostic. By integrating our learned model into simulation, we reduce the sim-to-real gap for test trajectories by 41%. RL-based controller trained with our model achieves a 50% improvement in fingertip pose tracking tasks on real tendon-driven robotic fingers. This approach is generalizable to different actuators and robot systems, and can enable RL policies to be used widely across tendon systems, advancing capabilities of dexterous manipulators and soft robots.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。