融合SLAM与SfM优势,实现长视频精准三维重建。
VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion

- 结合SLAM时序约束与SfM全局优化,提升重建稳定性。
- 在极端运动和视觉对称场景下,精度显著优于现有方法。
- 适合需要高精度重建的自动驾驶与机器人导航应用。
准确恢复任意非约束视频中相机的标定参数和度量位姿,将为导航与场景理解提供大规模训练数据。当前主流方法存在明显局限:SLAM因因果递增特性对初始化敏感,易受瞬时失败影响,且常为实时性过度优化,通常需已知相机标定;而传统SfM虽能实现最优初始化与全局优化,但缺乏对视觉对称性和极端运动的鲁棒性。为此,本文提出VidMap系统,融合SLAM的强时序约束与离线SfM的灵活性及全局优化能力,实现任意长、未标定视频的度量重建。该系统利用近期宽基线密集图像匹配进展,将时间顺序作为可靠回环闭合的关键依据,并引入度量单目深度先验增强全局优化。在涵盖极端运动和视觉对称性的多样化挑战数据集上,实验表明其性能显著优于最先进的经典或学习型SLAM与SfM方法,无论相机标定是否已知。代码已公开于https://github.com/cvg/vidmap。
原文摘要 · Abstract (English)
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。