用3D CAD模型增强6D姿态估计，提升机器人抓取精度
RAG-6DPose: Retrieval-Augmented 6D Pose Estimation via Leveraging CAD as Knowledge Base
- 用多视角渲染图和3D点构建多模态CAD知识库
- 通过视觉与几何特征检索匹配的CAD信息
- 适合需要高精度姿态估计的机器人应用
精确的6D姿态估计对机器人操作至关重要，可实现物体精确定位以完成抓取等任务。本文提出RAG-6DPose，一种利用3D CAD模型作为知识库的检索增强方法，融合视觉与几何线索。该方法包含三个阶段：1）通过多视角渲染图像提取2D视觉特征并附着3D点，构建多模态CAD知识库；2）基于查询图像，利用ReSPC模块从知识库中检索相关CAD特征；3）通过检索增强解码，将检索到的CAD信息融入姿态预测。在标准基准和真实机器人任务上的实验表明，该方法在处理遮挡和新视角时表现出色，具有优异的鲁棒性与有效性。补充材料见项目主页：https://sressers.github.io/RAG-6DPose。
原文摘要 · Abstract (English)
Accurate 6D pose estimation is key for robotic manipulation, enabling precise object localization for tasks like grasping. We present RAG-6DPose, a retrieval-augmented approach that leverages 3D CAD models as a knowledge base by integrating both visual and geometric cues. Our RAG-6DPose roughly contains three stages: 1) Building a Multi-Modal CAD Knowledge Base by extracting 2D visual features from multi-view CAD rendered images and also attaching 3D points; 2) Retrieving relevant CAD features from the knowledge base based on the current query image via our ReSPC module; and 3) Incorporating retrieved CAD information to refine pose predictions via retrieval-augmented decoding. Experimental results on standard benchmarks and real-world robotic tasks demonstrate the effectiveness and robustness of our approach, particularly in handling occlusions and novel viewpoints. Supplementary material is available on our project website: https://sressers.github.io/RAG-6DPose .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。