融合视觉与惯性/定位数据,提升复杂环境下的高精度实时定位建图能力
MASt3R-Fusion: Integrating Feed-Forward Visual Model with IMU, GNSS for High-Functionality SLAM
- 用前馈神经网络直接从图像生成点云,结合惯性与定位信息融合优化
- 在公开数据集上实现比现有系统更优的定位精度与鲁棒性
- 适合需要高可靠定位的自动驾驶与扩展现实应用
视觉SLAM是机器人、自动驾驶和扩展现实的核心技术,但传统系统在低纹理环境、尺度模糊及恶劣视觉条件下表现不佳。基于前馈神经网络的点云回归方法能利用学习的空间先验恢复高质量3D场景几何,突破传统多视图几何局限。然而,这类方法常忽略概率多传感器融合的优势。本文提出MASt3R-Fusion,一个将前馈点云回归与惯性测量单元(IMU)和全球导航卫星系统(GNSS)紧密融合的多传感器视觉SLAM框架。系统在通用度量尺度SE(3)因子图中引入基于Sim(3)的视觉对齐约束(黑塞形式),实现高效信息融合。采用分层因子图设计,支持实时滑动窗口优化与强回环闭合的全局优化,实现实时位姿追踪、度量尺度结构感知与全局一致建图。我们在公共基准和自采数据集上评估,结果表明该方法在精度与鲁棒性上显著优于现有以视觉为中心的多传感器SLAM系统。代码将开源,以支持复现与进一步研究(https://github.com/GREAT-WHU/MASt3R-Fusion)。
原文摘要 · Abstract (English)
Visual SLAM is a cornerstone technique in robotics, autonomous driving and extended reality (XR), yet classical systems often struggle with low-texture environments, scale ambiguity, and degraded performance under challenging visual conditions. Recent advancements in feed-forward neural network-based pointmap regression have demonstrated the potential to recover high-fidelity 3D scene geometry directly from images, leveraging learned spatial priors to overcome limitations of traditional multi-view geometry methods. However, the widely validated advantages of probabilistic multi-sensor information fusion are often discarded in these pipelines. In this work, we propose MASt3R-Fusion,a multi-sensor-assisted visual SLAM framework that tightly integrates feed-forward pointmap regression with complementary sensor information, including inertial measurements and GNSS data. The system introduces Sim(3)-based visualalignment constraints (in the Hessian form) into a universal metric-scale SE(3) factor graph for effective information fusion. A hierarchical factor graph design is developed, which allows both real-time sliding-window optimization and global optimization with aggressive loop closures, enabling real-time pose tracking, metric-scale structure perception and globally consistent mapping. We evaluate our approach on both public benchmarks and self-collected datasets, demonstrating substantial improvements in accuracy and robustness over existing visual-centered multi-sensor SLAM systems. The code will be released open-source to support reproducibility and further research (https://github.com/GREAT-WHU/MASt3R-Fusion).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。