稀疏帧下实现动态场景高保真重建，首次解决纹理丰富区域失真问题。
Sparse4DGS: 4D Gaussian Splatting for Sparse-Frame Dynamic Scene Reconstruction
- 针对稀疏帧设计纹理感知的变形正则化与优化机制
- 在多个数据集上显著优于现有动态或少样本方法
- 适合真实设备受限下的4D场景重建任务
动态高斯点阵方法在4D场景重建中表现优异，但依赖密集帧视频。现实中常因设备限制仅能获取稀疏帧。本文提出Sparse4DGS，首个面向稀疏帧动态场景重建的方法。观察发现，现有方法在稀疏帧下于标准空间与形变空间均失效，尤其在高纹理区域。Sparse4DGS通过聚焦纹理丰富区域来应对挑战：提出纹理感知形变正则化，引入基于纹理的深度对齐损失以约束高斯形变；提出纹理感知标准优化，将纹理相关噪声融入标准高斯梯度下降过程。大量实验表明，在使用稀疏帧输入时，该方法在NeRF-Synthetic、HyperNeRF、NeRF-DS及自建iPhone-4D数据集上均超越现有动态或少样本技术。
原文摘要 · Abstract (English)
Dynamic Gaussian Splatting approaches have achieved remarkable performance for 4D scene reconstruction. However, these approaches rely on dense-frame video sequences for photorealistic reconstruction. In real-world scenarios, due to equipment constraints, sometimes only sparse frames are accessible. In this paper, we propose Sparse4DGS, the first method for sparse-frame dynamic scene reconstruction. We observe that dynamic reconstruction methods fail in both canonical and deformed spaces under sparse-frame settings, especially in areas with high texture richness. Sparse4DGS tackles this challenge by focusing on texture-rich areas. For the deformation network, we propose Texture-Aware Deformation Regularization, which introduces a texture-based depth alignment loss to regulate Gaussian deformation. For the canonical Gaussian field, we introduce Texture-Aware Canonical Optimization, which incorporates texture-based noise into the gradient descent process of canonical Gaussians. Extensive experiments show that when taking sparse frames as inputs, our method outperforms existing dynamic or few-shot techniques on NeRF-Synthetic, HyperNeRF, NeRF-DS, and our iPhone-4D datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。