arXiv:2509.24370cs.CV2025-09

用视觉大模型提升点云配准,融合图像与几何信息

DINOReg: Strong Point Cloud Registration with Vision Foundation Model

  • 用DINOv2提取图像视觉特征,与几何特征在区块级融合
  • 在RGBD-3DMatch上配准召回率提升15.7%,内点比提高14.2%
  • 适合需要高精度点云配准的机器人、三维重建任务

点云配准是3D计算机视觉的基础任务。现有方法多仅依赖几何信息进行特征提取与匹配。近期研究虽引入了RGB-D数据中的颜色信息,但未充分挖掘图像中的纹理与语义信息,且特征融合存在图像信息损失,限制性能提升。本文提出DINOReg,一种充分利用视觉与几何信息的配准网络。受视觉基础模型启发,采用DINOv2从图像中提取丰富视觉特征,并在区块级别融合视觉与几何特征。该设计有效结合DINOv2提取的纹理与全局语义信息,以及几何主干捕捉的细节结构信息。同时,提出混合位置编码,融合图像空间与点云空间的位置信息,增强对区块间空间关系的感知能力。在RGBD-3DMatch和RGBD-3DLoMatch数据集上的大量实验表明,本方法显著优于当前最先进的纯几何与多模态配准方法,在配准召回率上提升15.7%,内点比例提升14.2%。代码已公开于https://github.com/ccjccjccj/DINOReg。

原文摘要 · Abstract (English)

Point cloud registration is a fundamental task in 3D computer vision. Most existing methods rely solely on geometric information for feature extraction and matching. Recently, several studies have incorporated color information from RGB-D data into feature extraction. Although these methods achieve remarkable improvements, they have not fully exploited the abundant texture and semantic information in images, and the feature fusion is performed in an image-lossy manner, which limit their performance. In this paper, we propose DINOReg, a registration network that sufficiently utilizes both visual and geometric information to solve the point cloud registration problem. Inspired by advances in vision foundation models, we employ DINOv2 to extract informative visual features from images, and fuse visual and geometric features at the patch level. This design effectively combines the rich texture and global semantic information extracted by DINOv2 with the detailed geometric structure information captured by the geometric backbone. Additionally, a mixed positional embedding is proposed to encode positional information from both image space and point cloud space, which enhances the model's ability to perceive spatial relationships between patches. Extensive experiments on the RGBD-3DMatch and RGBD-3DLoMatch datasets demonstrate that our method achieves significant improvements over state-of-the-art geometry-only and multi-modal registration methods, with a 14.2% increase in patch inlier ratio and a 15.7% increase in registration recall. The code is publicly available at https://github.com/ccjccjccj/DINOReg.

点云配准视觉大模型多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。