用不确定性正则化提升动态场景4D高斯点云重建效果
4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization
- 通过扩散模型与深度平滑先验，对观测稀疏区域施加选择性正则化
- 手持单目视频重建性能提升，静态场景少样本重建也有效
- 针对运动过快区域提出动态区域稠密化初始化方法
动态场景的新型视图合成在增强现实、虚拟现实等应用中日益重要。本文提出一种基于随意拍摄的单目视频进行动态场景4D高斯点云（4DGS）重建的新算法。为解决现有方法在真实视频中易过拟合的问题，引入不确定性感知正则化，识别观测稀疏区域，并对这些区域施加基于扩散模型和深度平滑性的额外先验。该方法同时提升了新视图合成性能与训练图像重建质量。此外，我们发现4DGS在快速运动区域因结构光流（SfM）无法提供可靠三维特征点而存在初始化问题。为此，提出利用估计深度图和场景流的动态区域稠密化初始化方法。实验表明，该方法显著改善了手持单目相机拍摄视频的4DGS重建效果，并在少样本静态场景重建中表现良好。
原文摘要 · Abstract (English)
Novel view synthesis of dynamic scenes is becoming important in various applications, including augmented and virtual reality. We propose a novel 4D Gaussian Splatting (4DGS) algorithm for dynamic scenes from casually recorded monocular videos. To overcome the overfitting problem of existing work for these real-world videos, we introduce an uncertainty-aware regularization that identifies uncertain regions with few observations and selectively imposes additional priors based on diffusion models and depth smoothness on such regions. This approach improves both the performance of novel view synthesis and the quality of training image reconstruction. We also identify the initialization problem of 4DGS in fast-moving dynamic regions, where the Structure from Motion (SfM) algorithm fails to provide reliable 3D landmarks. To initialize Gaussian primitives in such regions, we present a dynamic region densification method using the estimated depth maps and scene flow. Our experiments show that the proposed method improves the performance of 4DGS reconstruction from a video captured by a handheld monocular camera and also exhibits promising results in few-shot static scene reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。