arXiv:2411.18276cs.ROcs.AI2024-11ICRA被引 9

构建大规模零件级物体操作数据集,提升真实场景下机械臂灵活操控能力。

GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulation

  • 基于零件中心视角构建数据集,支持材质随机化与精细交互姿态标注。
  • 在仿真与真实场景中,显著提升深度感知与交互姿态预测性能。
  • 适合研究通用机器人操作、具身智能与3D视觉的开发者使用。

在家庭场景中有效操控关节物体是实现通用具身人工智能的关键步骤。主流3D视觉研究主要依赖深度感知与位姿检测,但在真实环境中常因透明盖子、反光把手等导致深度感知不准确。此外,现有方法普遍缺乏多样化的零件级交互方式,难以实现灵活适应。为此,我们提出了一个大规模零件级关节物体操作数据集,包含照片级真实感材质随机化与详细零件级、场景级可操作交互姿态标注。通过与多个先进深度估计和交互位姿预测方法结合,验证了该数据集的有效性。同时,我们提出一种新型模块化框架,在通用关节物体操控任务中表现更优且更鲁棒。大量实验表明,该数据集在仿真与真实场景中均显著提升了深度感知与可操作交互姿态预测的性能。

原文摘要 · Abstract (English)

Effectively manipulating articulated objects in household scenarios is a crucial step toward achieving general embodied artificial intelligence. Mainstream research in 3D vision has primarily focused on manipulation through depth perception and pose detection. However, in real-world environments, these methods often face challenges due to imperfect depth perception, such as with transparent lids and reflective handles. Moreover, they generally lack the diversity in part-based interactions required for flexible and adaptable manipulation. To address these challenges, we introduced a large-scale part-centric dataset for articulated object manipulation that features both photo-realistic material randomization and detailed annotations of part-oriented, scene-level actionable interaction poses. We evaluated the effectiveness of our dataset by integrating it with several state-of-the-art methods for depth estimation and interaction pose prediction. Additionally, we proposed a novel modular framework that delivers superior and robust performance for generalizable articulated object manipulation. Our extensive experiments demonstrate that our dataset significantly improves the performance of depth perception and actionable interaction pose prediction in both simulation and real-world scenarios. More information and demos can be found at: https://pku-epic.github.io/GAPartManip/.

机器人操控3D视觉数据集具身智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。