用2D扩散模型提升3D高斯点云的视角一致性，生成更清晰的全新视图。
3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
- 利用2D视频扩散模型建模视角间时序一致性，修复3D高斯渲染缺陷。
- 在稀疏视图下仍能生成无明显伪影的高质量新视角图像。
- 适合需要高保真3D重建与多视角一致性的场景建模任务。
新视角合成旨在从多张输入图像或视频中生成场景的新视图，近期的3D高斯点云（3DGS）方法已实现高效且逼真的渲染。然而，在输入视角稀疏等挑战性条件下，由于采样不足区域信息缺失，生成效果仍存在明显伪影。本文提出3DGS-Enhancer，一种增强3DGS表示质量的新流程。通过引入2D视频扩散先验，将复杂的3D视角一致性问题转化为视频生成中的时序一致性问题。该方法恢复新视角的视图一致潜在特征，并通过时空解码器将其与输入视图融合。增强后的视图用于微调初始3DGS模型，显著提升渲染性能。大规模无界场景数据集上的实验表明，3DGS-Enhancer在重建精度和渲染保真度上均优于当前最优方法。
原文摘要 · Abstract (English)
Novel-view synthesis aims to generate novel views of a scene from multiple input images or videos, and recent advancements like 3D Gaussian splatting (3DGS) have achieved notable success in producing photorealistic renderings with efficient pipelines. However, generating high-quality novel views under challenging settings, such as sparse input views, remains difficult due to insufficient information in under-sampled areas, often resulting in noticeable artifacts. This paper presents 3DGS-Enhancer, a novel pipeline for enhancing the representation quality of 3DGS representations. We leverage 2D video diffusion priors to address the challenging 3D view consistency problem, reformulating it as achieving temporal consistency within a video generation process. 3DGS-Enhancer restores view-consistent latent features of rendered novel views and integrates them with the input views through a spatial-temporal decoder. The enhanced views are then used to fine-tune the initial 3DGS model, significantly improving its rendering performance. Extensive experiments on large-scale datasets of unbounded scenes demonstrate that 3DGS-Enhancer yields superior reconstruction performance and high-fidelity rendering results compared to state-of-the-art methods. The project webpage is https://xiliu8006.github.io/3DGS-Enhancer-project .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。