用隐式神经场优化点云,让稀疏视角下3D/4D重建更清晰
SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction
- 将高斯点特征视为隐式神经场输出,增强空间相关性
- 在少视角条件下重建质量显著提升,动态场景也适用
- 适合做低资源3D建模、视频生成或实时渲染的研究者
从多视角图像中数字化静态3D场景和动态4D事件一直是计算机视觉与图形学中的长期挑战。近期,3D高斯溅射(3DGS)因其出色的重建质量、实时渲染能力及与常用可视化工具的兼容性,成为一种实用且可扩展的重建方法。然而,该方法需要大量输入视角才能实现高质量重建,带来显著的实践瓶颈,尤其在捕捉动态场景时,部署大规模摄像机阵列成本过高。本文发现,高斯点特征缺乏空间自相关性是导致3DGS在稀疏重建场景下表现不佳的关键因素之一。为此,我们提出一种优化策略,通过将点特征建模为对应隐式神经场的输出,有效正则化特征分布。实验表明,该方法在多种设置和场景复杂度下均能一致提升重建质量,适用于静态与动态场景。
原文摘要 · Abstract (English)
Digitizing 3D static scenes and 4D dynamic events from multi-view images has long been a challenge in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) has emerged as a practical and scalable reconstruction method, gaining popularity due to its impressive reconstruction quality, real-time rendering capabilities, and compatibility with widely used visualization tools. However, the method requires a substantial number of input views to achieve high-quality scene reconstruction, introducing a significant practical bottleneck. This challenge is especially severe in capturing dynamic scenes, where deploying an extensive camera array can be prohibitively costly. In this work, we identify the lack of spatial autocorrelation of splat features as one of the factors contributing to the suboptimal performance of the 3DGS technique in sparse reconstruction settings. To address the issue, we propose an optimization strategy that effectively regularizes splat features by modeling them as the outputs of a corresponding implicit neural field. This results in a consistent enhancement of reconstruction quality across various scenarios. Our approach effectively handles static and dynamic cases, as demonstrated by extensive testing across different setups and scene complexities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。