单阶段网络统一估计物体6D姿态与尺寸,提升机器人抓取精度。
CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Image
- 端到端融合RGB-D特征,直接预测零件类别、中心偏移和坐标图
- 在新数据集上达到91.3%的6D姿态准确率,优于现有方法
- 适合需要高精度抓取的机器人操作场景
本文针对机器人操作中刚性物体的类别级姿态估计问题,提出一种新的单阶段网络CAP-Net,用于从单张RGB-D图像中联合估计可动部件的6D姿态与尺寸。该方法结合RGB-D特征,端到端生成实例分割与NPCS表征。通过统一网络同时预测点级类别标签、中心偏移量和NPCS映射图,再利用聚类算法根据估计的中心距离分离各部件。最后将每个部件的NPCS区域对齐点云,恢复其最终姿态与尺寸。为缩小仿真到现实的差距,构建了目前最大的RGB-D可动物体数据集RGBD-Art,包含真实感图像与真实传感器模拟的深度噪声。在该数据集上的实验表明,本方法显著优于当前最优方法。实际机器人部署验证了其鲁棒性与出色的仿真到现实迁移能力,具有重要实用价值。代码与预训练模型已开源。
原文摘要 · Abstract (English)
This paper tackles category-level pose estimation of articulated objects in robotic manipulation tasks and introduces a new benchmark dataset. While recent methods estimate part poses and sizes at the category level, they often rely on geometric cues and complex multi-stage pipelines that first segment parts from the point cloud, followed by Normalized Part Coordinate Space (NPCS) estimation for 6D poses. These approaches overlook dense semantic cues from RGB images, leading to suboptimal accuracy, particularly for objects with small parts. To address these limitations, we propose a single-stage Network, CAP-Net, for estimating the 6D poses and sizes of Categorical Articulated Parts. This method combines RGB-D features to generate instance segmentation and NPCS representations for each part in an end-to-end manner. CAP-Net uses a unified network to simultaneously predict point-wise class labels, centroid offsets, and NPCS maps. A clustering algorithm then groups points of the same predicted class based on their estimated centroid distances to isolate each part. Finally, the NPCS region of each part is aligned with the point cloud to recover its final pose and size. To bridge the sim-to-real domain gap, we introduce the RGBD-Art dataset, the largest RGB-D articulated dataset to date, featuring photorealistic RGB images and depth noise simulated from real sensors. Experimental evaluations on the RGBD-Art dataset demonstrate that our method significantly outperforms the state-of-the-art approach. Real-world deployments of our model in robotic tasks underscore its robustness and exceptional sim-to-real transfer capabilities, confirming its substantial practical utility. Our dataset, code and pre-trained models are available on the project page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。