arXiv:2410.02237cs.CV2024-10NeurIPS被引 6

提出新方法Key-Grid,无监督检测3D关键点并保持语义一致性。

Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features

  • 用网格热图特征编码关键点关系,提升重建精度
  • 在多个数据集上关键点位置准确率与语义一致性达最优
  • 对噪声和下采样鲁棒,支持三维刚体与可变形物体

3D关键点检测在姿态估计、形状配准和机器人等领域广泛应用。现有无监督方法多针对刚性物体,面对可变形物体时关键点语义一致性差。本文提出新型无监督关键点检测器Key-Grid,基于自编码框架:编码器预测关键点,解码器利用关键点重构物体。创新性地引入3D网格热图(grid heatmap)——将均匀采样的3D立方空间中各网格点到关键点连线构成骨架的最短距离作为隐变量表示。同时将编码器各层信息融合至解码器。在多个基准数据集上评估表明,Key-Grid在关键点语义一致性和定位精度上均达到当前最优。此外,其对噪声和下采样具有强鲁棒性,并通过引入SE(3)不变骨干网络实现关键点的SE(3)不变性。

原文摘要 · Abstract (English)

Detecting 3D keypoints with semantic consistency is widely used in many scenarios such as pose estimation, shape registration and robotics. Currently, most unsupervised 3D keypoint detection methods focus on the rigid-body objects. However, when faced with deformable objects, the keypoints they identify do not preserve semantic consistency well. In this paper, we introduce an innovative unsupervised keypoint detector Key-Grid for both the rigid-body and deformable objects, which is an autoencoder framework. The encoder predicts keypoints and the decoder utilizes the generated keypoints to reconstruct the objects. Unlike previous work, we leverage the identified keypoint in formation to form a 3D grid feature heatmap called grid heatmap, which is used in the decoder section. Grid heatmap is a novel concept that represents the latent variables for grid points sampled uniformly in the 3D cubic space, where these variables are the shortest distance between the grid points and the skeleton connected by keypoint pairs. Meanwhile, we incorporate the information from each layer of the encoder into the decoder section. We conduct an extensive evaluation of Key-Grid on a list of benchmark datasets. Key-Grid achieves the state-of-the-art performance on the semantic consistency and position accuracy of keypoints. Moreover, we demonstrate the robustness of Key-Grid to noise and downsampling. In addition, we achieve SE-(3) invariance of keypoints though generalizing Key-Grid to a SE(3)-invariant backbone.

3D关键点无监督学习网格热图可变形物体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。