用扩散模型逐步优化匹配矩阵,提升图像与点云配准的鲁棒性。
Diff-Reg v2: Diffusion-Based Matching Matrix Estimation for Image Matching and 3D Registration
- 将匹配矩阵估计视为矩阵空间中的去噪过程,逐步优化对应关系。
- 在3D-3D、2D-3D和2D图像配准中均实现更优的匹配精度。
- 轻量级设计适合实际部署,适用于复杂形变和对称场景。
建立可靠的对应关系对所有配准任务(包括2D图像配准、3D点云配准及2D-3D图像到点云配准)至关重要。然而,尺度不一致、对称性和大形变等挑战常导致匹配模糊。以往基于特征或对应的方法依赖几何或语义特征生成或优化初始对应,部分方法引入特定几何先验(如拓扑保持性),但难以穷尽。此外,多数方法采用单步预测头,在复杂场景中易陷入局部最优。为此,本文提出一种创新范式:在匹配矩阵空间中使用扩散模型进行鲁棒匹配矩阵估计。模型将对应估计建模为矩阵空间中的去噪扩散过程,逐步精炼中间匹配矩阵至最优解。针对3D-3D和2D-3D任务,我们在双随机矩阵空间中应用扩散模型;对于2D图像配准,则在双Softmax投影正则化子空间中部署。三类任务均采用适配各自特性的自适应匹配矩阵嵌入,并保持统一的“匹配→形变”编码模式。同时,设计轻量化去噪模块。推理阶段,一旦提取并固定点或图像特征,该模块通过反向采样执行多步去噪预测。
原文摘要 · Abstract (English)
Establishing reliable correspondences is crucial for all registration tasks, including 2D image registration, 3D point cloud registration, and 2D-3D image-to-point cloud registration. However, these tasks are often complicated by challenges such as scale inconsistencies, symmetry, and large deformations, which can lead to ambiguous matches. Previous feature-based and correspondence-based methods typically rely on geometric or semantic features to generate or polish initial potential correspondences. Some methods typically leverage specific geometric priors, such as topological preservation, to devise diverse and innovative strategies tailored to a given enhancement goal, which cannot be exhaustively enumerated. Additionally, many previous approaches rely on a single-step prediction head, which can struggle with local minima in complex matching scenarios. To address these challenges, we introduce an innovative paradigm that leverages a diffusion model in matrix space for robust matching matrix estimation. Our model treats correspondence estimation as a denoising diffusion process in the matching matrix space, gradually refining the intermediate matching matrix to the optimal one. Specifically, we apply the diffusion model in the doubly stochastic matrix space for 3D-3D and 2D-3D registration tasks. In the 2D image registration task, we deploy the diffusion model in a matrix subspace where dual-softmax projection regularization is applied. For all three registration tasks, we provide adaptive matching matrix embedding implementations tailored to the specific characteristics of each task while maintaining a consistent "match-to-warp" encoding pattern. Furthermore, we adopt a lightweight design for the denoising module. In inference, once points or image features are extracted and fixed, this module performs multi-step denoising predictions through reverse sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。