用视觉状态空间模型实现单目图像与激光地图的跨模态定位匹配。
Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching
- 基于VMamba的双模态特征学习，通过多视角联合训练提升匹配精度。
- 在KITTI和KITTI-360上达到92.7%和88.5%的定位准确率，优于现有方法。
- 适合需要轻量级、高精度单目定位的自动驾驶系统部署。
在预先构建的激光雷达地图中实现单目相机定位可避免视觉SLAM系统的同步建图过程，降低自主定位的计算开销。核心挑战在于跨模态场景识别：根据在线获取的RGB图像从激光雷达点云地图中检索对应3D场景。本文提出一种高效框架，学习图像与点云的描述符，以视觉状态空间模型（VMamba）为骨干网络，采用像素-视角-场景联合训练策略进行跨模态对比学习。为解决视场差异问题，对点云从均匀分布的多个视角生成独立描述符，并设计可见3D点重叠策略，量化点云视角与图像间的相似性，实现多视角监督。在使用NetVLAD提取像素级特征时，补偿几何信息损失，并提出高效的多视角生成方案。在KITTI和KITTI-360数据集上的实验验证了方法的有效性与泛化能力。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Achieving monocular camera localization within pre-built LiDAR maps can bypass the simultaneous mapping process of visual SLAM systems, potentially reducing the computational overhead of autonomous localization. To this end, one of the key challenges is cross-modal place recognition, which involves retrieving 3D scenes (point clouds) from a LiDAR map according to online RGB images. In this paper, we introduce an efficient framework to learn descriptors for both RGB images and point clouds. It takes visual state space model (VMamba) as the backbone and employs a pixel-view-scene joint training strategy for cross-modal contrastive learning. To address the field-of-view differences, independent descriptors are generated from multiple evenly distributed viewpoints for point clouds. A visible 3D points overlap strategy is then designed to quantify the similarity between point cloud views and RGB images for multi-view supervision. Additionally, when generating descriptors from pixel-level features using NetVLAD, we compensate for the loss of geometric information, and introduce an efficient scheme for multi-view generation. Experimental results on the KITTI and KITTI-360 datasets demonstrate the effectiveness and generalization of our method. The code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。