用大模型融合视觉与激光数据,提升无结构环境下的定位闭环检测精度。
Multi-modal Loop Closure Detection with Foundation Models in Severely Unstructured Environments
- 基于Transformer大模型融合视觉与激光特征,实现多模态感知。
- 在低纹理区域的位姿估计精度显著提升,召回率优于现有方法。
- 适合火星等极端环境下的自主导航系统,可解释性强。
在缺乏卫星信号的环境中,如行星探测任务,鲁棒的回环检测是同步定位与地图构建(SLAM)的关键。传统视觉方法易受模糊和弱纹理影响,而激光雷达方法则受限于稀疏性和歧义性。本文提出MPRF多模态框架,利用基于Transformer的通用模型同时处理视觉与激光数据,在严重无结构环境中实现鲁棒回环检测。不同于仅依赖检索的方法,MPRF采用两阶段策略:先通过DINOv2特征结合SALAD聚合进行高效候选筛选,再以SONATA激光描述子进行几何验证,实现6-DoF位姿估计。在S3LI及S3LI Vulcano数据集上的实验表明,MPRF在精度上超越现有先进检索方法,并在低纹理区域显著增强位姿估计鲁棒性。通过提供适用于SLAM后端的可解释匹配关系,实现了精度、效率与可靠性的良好平衡,展示了基础模型在统一场景识别与位姿估计中的潜力。代码与模型将开源至github.com/DLR-RM/MPRF。
原文摘要 · Abstract (English)
Robust loop closure detection is a critical component of Simultaneous Localization and Mapping (SLAM) algorithms in GNSS-denied environments, such as in the context of planetary exploration. In these settings, visual place recognition often fails due to aliasing and weak textures, while LiDAR-based methods suffer from sparsity and ambiguity. This paper presents MPRF, a multimodal pipeline that leverages transformer-based foundation models for both vision and LiDAR modalities to achieve robust loop closure in severely unstructured environments. Unlike prior work limited to retrieval, MPRF integrates a two-stage visual retrieval strategy with explicit 6-DoF pose estimation, combining DINOv2 features with SALAD aggregation for efficient candidate screening and SONATA-based LiDAR descriptors for geometric verification. Experiments on the S3LI dataset and S3LI Vulcano dataset show that MPRF outperforms state-of-the-art retrieval methods in precision while enhancing pose estimation robustness in low-texture regions. By providing interpretable correspondences suitable for SLAM back-ends, MPRF achieves a favorable trade-off between accuracy, efficiency, and reliability, demonstrating the potential of foundation models to unify place recognition and pose estimation. Code and models will be released at github.com/DLR-RM/MPRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。