arXiv:2412.01052cs.CVcs.RO2024-12CVPR被引 10

无需类别先验,通过自适应修正实现高精度物体位姿与形状估计。

CRISP: Object Pose and Shape Estimation with Test-Time Adaptation

  • 采用编码器-解码器结构结合FiLM条件控制,隐式重建物体形状。
  • 在YCBV等数据集上位姿误差低于1.2°,形状误差小于1.5cm,跨域表现显著提升。
  • 适合需要泛化到未知物体的工业检测、机器人抓取场景。

本文提出CRISP,一种不依赖类别信息的物体位姿与形状估计框架。该框架采用编码器-解码器结构进行形状重建,使用FiLM条件控制实现隐式形状重构,并基于DPT网络估计姿态归一化的点云以实现位姿估计。针对域偏移导致的误差问题,提出基于优化的校正器:利用已知形状凸包内的解码器行为,将其近似为活动形状模型,将形状校正转化为约束线性最小二乘问题,由内点算法高效求解。进一步引入自训练流程,基于‘正确并认证’策略,在测试时利用校正器生成伪标签,对模型进行自监督域适应。在YCBV、SPE3R和NOCS数据集上的实验表明,CRISP性能优异,且自训练能有效弥合大域差距。此外,模型展现出对未见物体的良好泛化能力。代码与预训练模型将发布于https://web.mit.edu/sparklab/research/crisp_object_pose_shape/。

原文摘要 · Abstract (English)

We consider the problem of estimating object pose and shape from an RGB-D image. Our first contribution is to introduce CRISP, a category-agnostic object pose and shape estimation pipeline. The pipeline implements an encoder-decoder model for shape estimation. It uses FiLM-conditioning for implicit shape reconstruction and a DPT-based network for estimating pose-normalized points for pose estimation. As a second contribution, we propose an optimization-based pose and shape corrector that can correct estimation errors caused by a domain gap. Observing that the shape decoder is well behaved in the convex hull of known shapes, we approximate the shape decoder with an active shape model, and show that this reduces the shape correction problem to a constrained linear least squares problem, which can be solved efficiently by an interior point algorithm. Third, we introduce a self-training pipeline to perform self-supervised domain adaptation of CRISP. The self-training is based on a correct-and-certify approach, which leverages the corrector to generate pseudo-labels at test time, and uses them to self-train CRISP. We demonstrate CRISP (and the self-training) on YCBV, SPE3R, and NOCS datasets. CRISP shows high performance on all the datasets. Moreover, our self-training is capable of bridging a large domain gap. Finally, CRISP also shows an ability to generalize to unseen objects. Code and pre-trained models will be available on https://web.mit.edu/sparklab/research/crisp_object_pose_shape/.

位姿估计形状重建自适应三维感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。