用图像内在结构先验提升稀疏视角下的3D高斯点云重建质量。
DIP-GS: Deep Image Prior For Gaussian Splatting Sparse View Recovery
- 引入深度图像先验（DIP）约束，分层优化3D高斯参数以恢复缺失视角。
- 在稀疏视角下实现优于传统3DGS的重建效果，无需预训练模型或深度估计。
- 适合无标注数据、视角稀疏的3D场景重建任务，如旧照片修复与低采样场景建模。
3D高斯点云（3DGS）是一种领先的3D场景重建方法，能在实时渲染性能下实现高质量重建。其核心思想是将场景表示为一组3D高斯分布，并通过学习其参数拟合给定视角。尽管在多视角输入下表现优异，3DGS在稀疏视角重建中表现不佳，即输入视角稀少且覆盖不全、重叠度低。本文提出DIP-GS，一种基于深度图像先验（DIP）的3DGS表示方法。通过利用图像内部结构与模式，采用粗到精策略，DIP-GS可在传统3DGS失效的场景中运行，如稀疏视角恢复。本方法不依赖任何预训练模型（如生成模型或深度估计），仅使用输入帧。在多个稀疏视角重建任务中，DIP-GS达到当前最优（SOTA）水平，验证了其有效性。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) is a leading 3D scene reconstruction method, obtaining high-quality reconstruction with real-time rendering runtime performance. The main idea behind 3DGS is to represent the scene as a collection of 3D gaussians, while learning their parameters to fit the given views of the scene. While achieving superior performance in the presence of many views, 3DGS struggles with sparse view reconstruction, where the input views are sparse and do not fully cover the scene and have low overlaps. In this paper, we propose DIP-GS, a Deep Image Prior (DIP) 3DGS representation. By using the DIP prior, which utilizes internal structure and patterns, with coarse-to-fine manner, DIP-based 3DGS can operate in scenarios where vanilla 3DGS fails, such as sparse view recovery. Note that our approach does not use any pre-trained models such as generative models and depth estimation, but rather relies only on the input frames. Among such methods, DIP-GS obtains state-of-the-art (SOTA) competitive results on various sparse-view reconstruction tasks, demonstrating its capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。