用可微分引擎从真实视觉和控制信号中识别物体质量,实现高保真数字孪生与抓取强化。
D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
- 基于高斯点云构建可微分仿真引擎,联合优化物体质量与抓取策略。
- 在多种几何与质量物体上实现精准质量识别,提升抓取性能。
- 适合做机器人物理建模与少样本力觉抓取研究的团队使用。
仿真为机器人系统的数据生成与策略学习提供了低成本且灵活的平台,但仿真与真实世界动力学之间的差距仍是重大挑战,尤其在物理参数识别方面。本文提出一种真实-仿真-真实(real-to-sim-to-real)引擎,利用高斯点云表示构建可微分仿真环境,能够仅从真实世界的视觉观测和机器人控制信号中自动识别物体质量,并同时进行抓取策略学习。通过优化被操作物体的质量参数,该方法可自动生成高保真、物理合理的数字孪生体。此外,我们提出一种新方法,通过将可行的人类示范转化为仿真中的机器人示范,实现在有限数据下训练力觉感知的抓取策略。大量实验表明,该引擎在多种物体几何形状与质量条件下均能实现准确且鲁棒的质量识别;优化后的质量参数显著提升了力觉抓取策略的性能,有效缩小了仿真到现实的差距。
原文摘要 · Abstract (English)
Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant challenge, especially in physical parameter identification. In this work, we introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine, enabling object mass identification from real-world visual observations and robot control signals, while enabling grasping policy learning simultaneously. Through optimizing the mass of the manipulated object, our method automatically builds high-fidelity and physically plausible digital twins. Additionally, we propose a novel approach to train force-aware grasping policies from limited data by transferring feasible human demonstrations into simulated robot demonstrations. Through comprehensive experiments, we demonstrate that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values. Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping, effectively reducing the sim-to-real gap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。