用RGB-D数据提升6D位姿估计的鲁棒性，应对光照、遮挡等挑战。
VLM6D: VLM based 6Dof Pose Estimation based on RGB-D Images
- 双流架构：RGB用DINOv2，深度图用PointNet++，互补特征融合
- 在Occluded-LineMOD上达到新SOTA，对遮挡和纹理缺失更稳定
- 适合需要高精度位姿估计的机器人抓取、AR应用
计算机视觉中精确计算6D物体位姿是主要挑战，现有方法在真实场景中常因光照变化、无纹理物体和严重遮挡而表现脆弱。为此，VLM6D提出一种新型双流架构，利用RGB-D输入中视觉与几何数据的互补优势实现鲁棒且精确的位姿估计。框架采用强大的自监督视觉变换器（DINOv2）处理RGB图像，借助其丰富的视觉语法先验，在纹理和光照变化下仍具强鲁棒性；同时使用PointNet++编码器处理由深度图生成的3D点云，实现对稀疏、碎片化数据的稳健几何推理。两条特征流经有效融合后输入多任务预测头。大量实验表明，VLM6D在具有挑战性的Occluded-LineMOD数据集上取得新的最先进性能，验证了其优越的鲁棒性与准确性。
原文摘要 · Abstract (English)
The primary challenge in computer vision is precisely calculating the pose of 6D objects, however many current approaches are still fragile and have trouble generalizing from synthetic data to real-world situations with fluctuating lighting, textureless objects, and significant occlusions. To address these limitations, VLM6D, a novel dual-stream architecture that leverages the distinct strengths of visual and geometric data from RGB-D input for robust and precise pose estimation. Our framework uniquely integrates two specialized encoders: a powerful, self-supervised Vision Transformer (DINOv2) processes the RGB modality, harnessing its rich, pre-trained understanding of visual grammar to achieve remarkable resilience against texture and lighting variations. Concurrently, a PointNet++ encoder processes the 3D point cloud derived from depth data, enabling robust geometric reasoning that excels even with the sparse, fragmented data typical of severe occlusion. These complementary feature streams are effectively fused to inform a multi task prediction head. We demonstrate through comprehensive experiments that VLM6D obtained new SOTA performance on the challenging Occluded-LineMOD, validating its superior robustness and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。