arXiv:2510.13310cs.CV2025-10被引 3

打造首个全GPU的SfM系统,实现深度学习时代的高效三维重建。

InstantSfM: Towards GPU-Native SfM for the Deep Learning Era

  • 全GPU架构,兼容PyTorch,支持深度学习流水线无缝集成。
  • 引入深度约束的雅可比结构,解决尺度模糊问题,精度媲美传统方法。
  • 在大规模场景下速度提升40倍,适合实时3D重建与神经场景建模。

Structure-from-Motion(SfM)是从多视角图像中恢复相机位姿与场景结构的基础技术,是3D重建及现代神经场景表示(如3D高斯溅射)的关键前置模块。然而,现有成熟SfM系统仍以CPU为中心,依赖传统优化工具链,难以匹配当前基于GPU的、学习驱动的流水线,限制了大规模场景下的可扩展性。尽管近年已有GPU加速的束调整(BA)研究展示了并行稀疏优化潜力,但构建完整全局SfM系统仍面临度量尺度恢复与数值鲁棒性等未解难题。本文提出全GPU化的、兼容PyTorch的全局SfM系统InstantSfM,通过在全局定位与束调整中嵌入度量深度先验,构建深度约束的雅可比结构,从而在优化框架内解决尺度模糊问题。为保障数值稳定性,采用优化的GPU友好方式对雅可比矩阵中的欠约束变量进行显式过滤。在多样数据集上的大量实验表明,InstantSfM在保持与经典流水线及近期学习方法相当的重建精度的同时,实现最先进效率,大规模场景下相较COLMAP提速约40倍。

原文摘要 · Abstract (English)

Structure-from-Motion (SfM) is a fundamental technique for recovering camera poses and scene structure from multi-view imagery, serving as a critical upstream component for applications ranging from 3D reconstruction to modern neural scene representations such as 3D Gaussian Splatting. However, most mature SfM systems remain CPU-centric and built upon traditional optimization toolchains, creating a growing mismatch with modern GPU-based, learning-driven pipelines and limiting scalability in large-scale scenes. While recent advances in GPU-accelerated bundle adjustment (BA) have demonstrated the potential of parallel sparse optimization, extending these techniques to build a complete global SfM system remains challenging due to unresolved issues in metric scale recovery and numerical robustness. In this paper, we implement a fully GPU-based and PyTorch-compatible global SfM system, named InstantSfM, to integrate seamlessly with modern learning pipelines. InstantSfM embeds metric depth priors directly into both global positioning and BA through a depth-constrained Jacobian structure, thereby resolving scale ambiguity within the optimization framework. To ensure numerical stability, we employ explicit filtering of under-constrained variables for the Jacobian matrix in an optimized GPU-friendly manner. Extensive experiments on diverse datasets demonstrate that InstantSfM achieves state-of-the-art efficiency while maintaining reconstruction accuracy comparable to both established classical pipelines and recent learning-based methods, showing up to ${\sim40\times}$ speedup over COLMAP on large-scale scenes.

三维重建GPU加速深度先验视觉定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。