通过真实数据反推仿真参数，提升机械臂力控策略的现实迁移能力。
Enhancing Sim2Real Transfer for Torque-Controlled Robots through Real2Sim Dynamics Estimation and Reinforcement Learning
- 用遗传算法优化仿真中的摩擦、惯量和重力参数，使真实与仿真轨迹匹配。
- 在7自由度Franka Panda机械臂上，转移成功率显著提升，多任务表现更稳定。
- 适合做力控机器人仿真到现实部署的研究者参考。
将强化学习策略从仿真迁移到真实机器人仍是一大挑战，尤其在低层力矩控制场景下，微小建模误差即可导致不稳定或不安全行为。本文提出一种Real2Sim2Real流程，结合轨迹匹配、遗传算法参数优化与领域随机化，针对7自由度Franka Emika Panda机械臂，通过最小化真实与仿真关节轨迹误差，识别出摩擦、惯量和重力补偿参数。基于校准后的动力学模型，在仿真中训练基于TQC的强化学习智能体。该策略在Gazebo和MuJoCo环境中验证后，成功部署于真实机器人。结果表明，参数调优后跟踪精度和策略鲁棒性显著提升，实现了多个目标达成果的平滑仿真到现实迁移。本工作强调了精确物理建模对实现稳定可泛化的力控强化学习策略的关键作用。
原文摘要 · Abstract (English)
Transferring reinforcement learning policies from simulation to Real-World robots remains a major challenge, particularly when dealing with low-level torque control, where even small modelling inaccuracies can lead to unstable or unsafe behaviours. In this work, we propose a Real2Sim2Real pipeline that improves Sim2Real transfer for torque-controlled robotic arms by combining trajectory matching, parameter optimization via genetic algorithms, and domain randomization. Using the 7-DOF Franka Emika Panda robot, we first identify friction, inertia, and gravity compensation parameters by minimizing the error between real and simulated joint trajectories. These calibrated dynamics are then used to train a TQC-based reinforcement learning agent in simulation. The trained policy is evaluated in both Gazebo and MuJoCo environments, and finally deployed on the real robot. Our results demonstrate a significant improvement in tracking accuracy and policy robustness after parameter tuning, with smooth policy transfer from simulation to the Real-World across multiple target-reaching tasks. This work highlights the effectiveness of accurate physical modelling in enabling stable and generalizable torque-based reinforcement learning policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。