arXiv:2411.01200cs.ROcs.AI2024-11NeurIPS被引 33

构建真实衣物操作仿真与评测平台,解决机器人抓取衣物的通用性难题。

GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation

  • 融合多种物理模拟方法,构建高保真衣物交互环境
  • 覆盖多样服装类型与任务,验证算法在复杂场景下的表现
  • 开源工具包助力研究者突破仿真到现实的差距

衣物与柔性物体的操作是家庭服务机器人研发中的关键挑战。由于其复杂的动力学和拓扑结构,现有方法受限于基准测试多样性不足和仿真不真实的问题。为此,我们提出GarmentLab,一个内容丰富的仿真与评测平台,支持多种服装类型、机械臂及操作任务。平台涵盖衣物、柔性体、刚体、流体与人体之间的交互,结合有限元法(FEM)与位置基于动力学(PBD)等多模态模拟技术,并引入仿真实现到现实的迁移算法与真实世界评测。我们在该平台上评估了先进的视觉、强化学习与模仿学习方法,揭示当前算法在泛化能力上的显著局限。开源环境与全面分析为未来衣物操作研究提供了有力支撑。代码将尽快公开。

原文摘要 · Abstract (English)

Manipulating garments and fabrics has long been a critical endeavor in the development of home-assistant robots. However, due to complex dynamics and topological structures, garment manipulations pose significant challenges. Recent successes in reinforcement learning and vision-based methods offer promising avenues for learning garment manipulation. Nevertheless, these approaches are severely constrained by current benchmarks, which offer limited diversity of tasks and unrealistic simulation behavior. Therefore, we present GarmentLab, a content-rich benchmark and realistic simulation designed for deformable object and garment manipulation. Our benchmark encompasses a diverse range of garment types, robotic systems and manipulators. The abundant tasks in the benchmark further explores of the interactions between garments, deformable objects, rigid bodies, fluids, and human body. Moreover, by incorporating multiple simulation methods such as FEM and PBD, along with our proposed sim-to-real algorithms and real-world benchmark, we aim to significantly narrow the sim-to-real gap. We evaluate state-of-the-art vision methods, reinforcement learning, and imitation learning approaches on these tasks, highlighting the challenges faced by current algorithms, notably their limited generalization capabilities. Our proposed open-source environments and comprehensive analysis show promising boost to future research in garment manipulation by unlocking the full potential of these methods. We guarantee that we will open-source our code as soon as possible. You can watch the videos in supplementary files to learn more about the details of our work. Our project page is available at: https://garmentlab.github.io/

机器人操作柔性体仿真仿真实验衣物生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。