arXiv:2410.08743cs.CV2024-10中稿 · IROS 2024被引 10

无需精确相机位姿即可实现快速3D场景重建与新视角合成。

Look Gauss, No Pose: Novel View Synthesis using Gaussian Splatting without Accurate Pose Initialization

  • 联合优化几何结构与相机位姿,通过光度残差反向传播求解梯度。
  • 在真实场景和复杂轨迹上实现快速收敛,精度达最新水平。
  • 适合缺乏准确位姿数据的现实应用,如移动设备摄影或无人机拍摄。

3D高斯点阵近年来成为从一组带位姿的输入图像中快速准确生成新视角的强大工具。然而,如同大多数新视角合成方法,它依赖于精确的相机位姿信息,限制了其在实际场景中的应用,因为在这些场景中获取精确位姿可能困难甚至不可能。本文提出对3D高斯点阵框架的扩展,通过光度残差优化相机外参。我们推导出解析梯度,并将其与现有的高性能CUDA实现集成。该方法支持下游任务如6-自由度相机位姿估计及联合重建与相机精修。特别地,我们在真实场景中实现了快速收敛和高精度位姿估计。本方法可在无需准确位姿的前提下实现快速3D场景重建,同时在新视角合成上达到最先进水平。相比多数竞争方法,优化速度显著更快,渲染速度提升数倍。我们在真实场景和模拟环境中的复杂轨迹上进行了验证,在LLFF基准上取得最优结果,运行时间比最高效的竞争方法减少2至4倍。源代码将发布于https://github.com/Schmiddo/noposegs。

原文摘要 · Abstract (English)

3D Gaussian Splatting has recently emerged as a powerful tool for fast and accurate novel-view synthesis from a set of posed input images. However, like most novel-view synthesis approaches, it relies on accurate camera pose information, limiting its applicability in real-world scenarios where acquiring accurate camera poses can be challenging or even impossible. We propose an extension to the 3D Gaussian Splatting framework by optimizing the extrinsic camera parameters with respect to photometric residuals. We derive the analytical gradients and integrate their computation with the existing high-performance CUDA implementation. This enables downstream tasks such as 6-DoF camera pose estimation as well as joint reconstruction and camera refinement. In particular, we achieve rapid convergence and high accuracy for pose estimation on real-world scenes. Our method enables fast reconstruction of 3D scenes without requiring accurate pose information by jointly optimizing geometry and camera poses, while achieving state-of-the-art results in novel-view synthesis. Our approach is considerably faster to optimize than most competing methods, and several times faster in rendering. We show results on real-world scenes and complex trajectories through simulated environments, achieving state-of-the-art results on LLFF while reducing runtime by two to four times compared to the most efficient competing method. Source code will be available at https://github.com/Schmiddo/noposegs .

3D重建新视角合成位姿优化高斯点阵

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。