DeformX实现细长柔性物体高保真物理与视觉仿真,支持机器人操控学习。
DeformX: A Versatile Co-Simulation Framework for Deformable Linear Objects

- 融合柯西杆物理引擎与Isaac Sim,实现柔性线状物的精准动力学模拟。
- 生成数据训练的分割模型在真实图像上mAP@75提升10.2%,实测抓取误差仅6.6cm。
- 适用于机器人操控策略训练与合成数据生成,具强模拟到现实迁移能力。
细长可变形物体(如电线、电缆、绳索)在机器人操作中常见,但同时实现视觉真实感与物理准确性仍具挑战。现有视觉仿真方法依赖程序化几何体,缺乏物理驱动的形变行为;而基于物理的方法常将DLO近似为刚性链或通用软体,无法准确捕捉其弯曲、扭转和剪切力学特性。本文提出DeformX,一个集成专用柯西杆物理引擎与NVIDIA Isaac Sim的联合仿真框架,可实现兼具物理真实性和视觉逼真度的DLO模拟。其柯西杆引擎支持动态模拟与自碰撞,以及与任意自由形态网格的接触交互。为实现高保真可视化,采用网格蒙皮技术将离散杆体形变映射至导入的CAD模型。据我们所知,DeformX是首个统一真实视觉、原理性物理与机器人学习兼容性的DLO仿真框架。我们展示了其在合成数据生成与策略学习中的多功能性,并通过与真实实验对比验证了其视觉与物理保真度。值得注意的是,基于DeformX生成数据微调的Segment Anything Model 3(SAM3),在真实图像上的线段分割mAP@75提升10.2%;完全在DeformX中训练的绳索摆动策略,在真实UR5e机械臂上实现平均命中误差6.6cm,显著体现其优秀的模拟到现实迁移能力。
原文摘要 · Abstract (English)
Deformable linear objects (DLOs) such as wires, cables, and ropes are common in robotic manipulation tasks, yet simulating them with both visual realism and physical accuracy remains challenging. Existing visual simulation methods typically rely on procedural geometric primitives that lack physically grounded deformation behavior, while physics-based approaches with robot learning support often approximate DLOs as rigid-link chains or generic soft bodies, failing to accurately capture the bending, twisting, and shear mechanics of slender elastic structures. In this work, we introduce DeformX, a co-simulation framework that integrates a dedicated Cosserat rod physics engine with NVIDIA Isaac Sim, enabling DLO simulations that are both physically faithful and visually realistic. Our Cosserat rod engine simulates the dynamics and self-collisions of DLOs, and contact interactions with arbitrary free-form meshes. To achieve high-fidelity visualization, we employ mesh skinning to map discrete rod deformations onto imported CAD models. To the best of our knowledge, DeformX is the one of the first frameworks for DLO simulation that unifies realistic visualization, principled physics, and compatibility with robot learning pipelines. We demonstrate its versatility across synthetic data generation and policy learning for DLO manipulation, and validate visual and physical fidelity through comparisons against real-world experiments. Notably, fine-tuning Segment Anything Model 3 (SAM3) on DeformX-generated data yields a 10.2% mAP@75 improvement in real-image wire segmentation, and a rope-swinging policy trained entirely in DeformX achieves a mean target-hitting error of 6.6 cm on a UR5e manipulator in real-world trials, highlighting its strong sim-to-real transfer capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。