arXiv:2510.11687cs.CV2025-10被引 1

无需模板或类别标签,单视角即可同时估计物体6D姿态、尺寸和形状。

Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View

  • 用视觉基础模型与3D点云融合,通过Transformer+专家混合实现统一建模。
  • 在149个类别的合成数据上训练,实测4个真实场景基准上零样本泛化表现最优。
  • 适合机器人抓取、具身智能等需要开放集物体理解的场景。

从视觉输入中估计物体的6D姿态、尺寸和形状是计算机视觉中的基础问题,对机器人抓取与操作至关重要。现有方法要么依赖特定物体的先验(如CAD模型或模板),要么因姿态-形状耦合及多阶段流程导致类别间泛化能力有限。本文提出一种统一的、类别无关的框架,仅需单张RGB-D图像即可同时预测6D姿态、尺寸和稠密形状,无需测试时使用模板、CAD模型或类别标签。模型将视觉基础模型的密集2D特征与部分3D点云融合,采用增强型Transformer编码器与专家混合机制,并使用并行解码器分别完成姿态-尺寸估计与形状重建,实现28 FPS的实时推理。该框架仅在包含149个类别的SOPE数据集合成数据上训练,评估覆盖四个多样化基准:SOPE、ROPE、ObjaversePose和HANDAL,涵盖超300个类别。在已见类别上达到当前最优准确率,并在未见过的真实物体上展现出显著的零样本泛化能力,为机器人与具身智能中的开放集6D理解树立新标准。

原文摘要 · Abstract (English)

Estimating an object's 6D pose, size, and shape from visual input is a fundamental problem in computer vision, with critical applications in robotic grasping and manipulation. Existing methods either rely on object-specific priors such as CAD models or templates, or suffer from limited generalization across categories due to pose-shape entanglement and multi-stage pipelines. In this work, we propose a unified, category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image, without requiring templates, CAD models, or category labels at test time. Our model fuses dense 2D features from vision foundation models with partial 3D point clouds using a Transformer encoder enhanced by a Mixture-of-Experts, and employs parallel decoders for pose-size estimation and shape reconstruction, achieving real-time inference at 28 FPS. Trained solely on synthetic data from 149 categories in the SOPE dataset, our framework is evaluated on four diverse benchmarks SOPE, ROPE, ObjaversePose, and HANDAL, spanning over 300 categories. It achieves state-of-the-art accuracy on seen categories while demonstrating remarkably strong zero-shot generalization to unseen real-world objects, establishing a new standard for open-set 6D understanding in robotics and embodied AI.

6D姿态估计物体形状重建零样本泛化机器人抓取

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。