用基础模型+几何编码，端到端搞定物体6D姿态估计
OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding
- 基于Transformer和基础模型提取特征，融合多视角模板
- 引入NOCS编码显式3D几何先验，提升姿态估计精度
- 支持从CAD模型到神经重建的灵活适配，适合工业场景
我们提出一个统一的端到端框架，无缝集成物体检测与姿态估计，并具备通用的上手流程。系统首先通过传统3D CAD模型或从多视图图像快速重建高保真神经表示（NeRF）生成物体表征。测试时，先用CNOS检测器定位目标物体，再由新型姿态估计模块OPFormer推断精确的6D姿态。OPFormer核心为基于Transformer的架构，利用基础模型进行鲁棒特征提取，通过联合编码多个模板视图并结合归一化物体坐标空间（NOCS）显式引入3D几何先验，增强特征表达。解码器建立鲁棒的2D-3D对应关系以确定最终姿态。在BOP基准测试中，该系统在准确率与效率间取得良好平衡，展示了在基于模型与无模型场景下的实用潜力。
原文摘要 · Abstract (English)
We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation with a versatile onboarding process. Our pipeline begins with an onboarding stage that generates object representations from either traditional 3D CAD models or, in their absence, by rapidly reconstructing a high-fidelity neural representation (NeRF) from multi-view images. Given a test image, our system first employs the CNOS detector to localize target objects. For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose. The core of OPFormer is a transformer-based architecture that leverages a foundation model for robust feature extraction. It uniquely learns a comprehensive object representation by jointly encoding multiple template views and enriches these features with explicit 3D geometric priors using Normalized Object Coordinate Space (NOCS). A decoder then establishes robust 2D-3D correspondences to determine the final pose. Evaluated on the challenging BOP benchmarks, our integrated system demonstrates a strong balance between accuracy and efficiency, showcasing its practical applicability in both model-based and model-free scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。