端到端实时估计多物体6自由度位姿,精度超越现有单模型方法。
SEMPose: A Single End-to-end Network for Multi-object Pose Estimation
- 设计纹理-形状引导的特征金字塔网络,应对物体尺寸差异。
- 迭代精修头分步回归旋转与平移,提升位姿估计精度。
- 适合需要高精度、低延迟的多物体位姿场景,如机器人抓取。
在计算机视觉中,从RGB图像估计六自由度位姿是基础任务,但在多物体场景下极具挑战。现有最优方法通常采用间接策略:先匹配2D-3D点,再用PnP求解,但无法端到端训练。直接方法因物体尺寸变化和遮挡等问题,精度较低。为此,本文提出SEMPose,一种端到端的多物体位姿估计网络。它采用纹理-形状引导的特征金字塔网络,有效缓解物体尺寸差异问题;并引入迭代精修头,分步回归旋转和平移,逐步提升精度。训练时通过选取可见部分正样本,减轻遮挡影响。实验表明,SEMPose可在32 FPS下仅依赖RGB图像完成推理,且推理时间不随目标数量增加而增长。在LM-O和YCB-V数据集上,其性能优于其他基于RGB的单模型方法,即使对比使用额外优化的多模型方法,仍具竞争力。
原文摘要 · Abstract (English)
In computer vision, estimating the six-degree-of-freedom pose from an RGB image is a fundamental task. However, this task becomes highly challenging in multi-object scenes. Currently, the best methods typically employ an indirect strategy, which identifies 2D and 3D correspondences, and then solves with the Perspective-n-Points method. Yet, this approach cannot be trained end-to-end. Direct methods, on the other hand, suffer from lower accuracy due to challenges such as varying object sizes and occlusions. To address these issues, we propose SEMPose, an end-to-end multi-object pose estimation network. SEMPose utilizes a well-designed texture-shape guided feature pyramid network, effectively tackling the challenge of object size variations. Additionally, it employs an iterative refinement head structure, progressively regressing rotation and translation separately to enhance estimation accuracy. During training, we alleviate the impact of occlusion by selecting positive samples from visible parts. Experimental results demonstrate that SEMPose can perform inference at 32 FPS without requiring inputs other than the RGB image. It can accurately estimate the poses of multiple objects in real time, with inference time unaffected by the number of target objects. On the LM-O and YCB-V datasets, our method outperforms other RGB-based single-model methods, achieving higher accuracy. Even when compared with multi-model methods and approaches that use additional refinement, our results remain competitive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。