无需相机位姿和SfM预处理，通过分层训练实现高质量视频3D高斯点渲染。
SfM-Free 3D Gaussian Splatting via Hierarchical Training
- 分层训练多个局部高斯模型并融合为全局统一表示
- 在Tanks and Temples上平均提升2.25dB PSNR，最高达3.72dB
- 适合无位姿信息的视频三维重建，尤其适用于大运动场景
标准3D高斯点渲染依赖已知或预计算的相机位姿及由结构光复原（SfM）获得的稀疏点云进行初始化与扩展。本文提出一种新型无SfM的3D高斯点渲染方法（SFGS），适用于视频输入，无需预先知道相机位姿或执行SfM预处理。该方法引入分层训练策略，对特定场景区域优化多个3D高斯表示，并将其合并为一个统一的3DGS模型。为应对大范围相机运动，利用视频帧插值模型增强时序一致性。同时，采用多源监督以减少过拟合并提升表达能力。实验结果表明，本方法显著优于现有SfM-free新视角合成技术：在Tanks and Temples数据集上，平均PSNR提升2.25dB，最佳场景达3.72dB；在CO3D-V2数据集上，平均提升1.74dB，最高达3.90dB。代码已公开于https://github.com/jibo27/3DGS_Hierarchical_Training。
原文摘要 · Abstract (English)
Standard 3D Gaussian Splatting (3DGS) relies on known or pre-computed camera poses and a sparse point cloud, obtained from structure-from-motion (SfM) preprocessing, to initialize and grow 3D Gaussians. We propose a novel SfM-Free 3DGS (SFGS) method for video input, eliminating the need for known camera poses and SfM preprocessing. Our approach introduces a hierarchical training strategy that trains and merges multiple 3D Gaussian representations -- each optimized for specific scene regions -- into a single, unified 3DGS model representing the entire scene. To compensate for large camera motions, we leverage video frame interpolation models. Additionally, we incorporate multi-source supervision to reduce overfitting and enhance representation. Experimental results reveal that our approach significantly surpasses state-of-the-art SfM-free novel view synthesis methods. On the Tanks and Temples dataset, we improve PSNR by an average of 2.25dB, with a maximum gain of 3.72dB in the best scene. On the CO3D-V2 dataset, we achieve an average PSNR boost of 1.74dB, with a top gain of 3.90dB. The code is available at https://github.com/jibo27/3DGS_Hierarchical_Training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。