用异构图提升图像与点云配准的跨域泛化能力
Hg-I2P: Bridging Modalities for Generalizable Image-to-Point-Cloud Registration via Heterogeneous Graphs
- 构建图像与点云间的异构图，统一优化特征与对应关系
- 在6个跨域场景中准确率显著超越现有方法
- 适合做3D感知、机器人定位等需要跨模态对齐的研究者
图像到点云（I2P）配准旨在通过建立可靠的2D-3D对应关系，将二维图像与三维点云对齐。由于图像与点云间存在巨大模态差异，难以学习既具区分性又具泛化性的特征，导致在未见场景中性能严重下降。本文提出一种异构图结构，可在统一架构中同时优化跨模态特征与对应关系。该图表示分段后的2D与3D区域间的映射，增强跨模态特征交互，提升特征区分度；同时通过建模图中顶点与边的一致性，实现不可靠对应关系的剔除。基于此，提出Hg-I2P方法：通过挖掘多路径特征关系构建异构图，利用异构边引导特征自适应，并基于图投影一致性进行对应关系剪枝。在六个室内外基准数据集的跨域设置下实验表明，Hg-I2P在泛化性和准确性上均显著优于现有方法。代码已开源。
原文摘要 · Abstract (English)
Image-to-point-cloud (I2P) registration aims to align 2D images with 3D point clouds by establishing reliable 2D-3D correspondences. The drastic modality gap between images and point clouds makes it challenging to learn features that are both discriminative and generalizable, leading to severe performance drops in unseen scenarios. We address this challenge by introducing a heterogeneous graph that enables refining both cross-modal features and correspondences within a unified architecture. The proposed graph represents a mapping between segmented 2D and 3D regions, which enhances cross-modal feature interaction and thus improves feature discriminability. In addition, modeling the consistency among vertices and edges within the graph enables pruning of unreliable correspondences. Building on these insights, we propose a heterogeneous graph embedded I2P registration method, termed Hg-I2P. It learns a heterogeneous graph by mining multi-path feature relationships, adapts features under the guidance of heterogeneous edges, and prunes correspondences using graph-based projection consistency. Experiments on six indoor and outdoor benchmarks under cross-domain setups demonstrate that Hg-I2P significantly outperforms existing methods in both generalization and accuracy. Code is released on https://github.com/anpei96/hg-i2p-demo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。