无需相机位姿即可重建360°场景，实现逼真新视角生成。
Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors
- 利用高斯内部深度先验，从无位姿视频中恢复相机姿态。
- 通过深度一致性的筛选，高效提升高斯点密度与渲染质量。
- 适合缺乏相机位姿的全景视频重建任务，尤其适用于移动设备采集场景。
全景3D高斯点云拼贴是360度场景表示的关键技术，现有方法通常依赖耗时的运动恢复结构（SfM）获取相机位姿与稀疏点先验。本文提出一种无位姿全景3DGS方法PFGS360，可直接从无位姿全景视频重建3D高斯。为实现精准相机位姿估计，我们构建球面一致性感知的位姿估计模块，通过重建高斯与未标定图像间的2D-3D对应关系，利用高斯内部深度先验进行恢复。此外，引入深度内点感知的稠密化模块，结合单目深度先验提取深度内点与高斯异常点，实现高效稠密化，显著提升新视角合成保真度。实验表明，该方法在真实与合成360度视频上均显著优于现有无位姿及有位姿3DGS方法。代码已开源：https://github.com/zcq15/PFGS360。
原文摘要 · Abstract (English)
Omnidirectional 3D Gaussian Splatting with panoramas is a key technique for 3D scene representation, and existing methods typically rely on slow SfM to provide camera poses and sparse points priors. In this work, we propose a pose-free omnidirectional 3DGS method, named PFGS360, that reconstructs 3D Gaussians from unposed omnidirectional videos. To achieve accurate camera pose estimation, we first construct a spherical consistency-aware pose estimation module, which recovers poses by establishing consistent 2D-3D correspondences between the reconstructed Gaussians and the unposed images using Gaussians' internal depth priors. Besides, to enhance the fidelity of novel view synthesis, we introduce a depth-inlier-aware densification module to extract depth inliers and Gaussian outliers with consistent monocular depth priors, enabling efficient Gaussian densification and achieving photorealistic novel view synthesis. The experiments show significant outperformance over existing pose-free and pose-aware 3DGS methods on both real-world and synthetic 360-degree videos. Code is available at https://github.com/zcq15/PFGS360.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。