用NeRF生成高质量图像，提升3D高斯溅射的街景渲染效果。
Leveraging NeRF-Rendered Images for 3D Gaussian Splatting
- 用预训练的街景NeRF生成训练图像，指导3D高斯溅射
- 通过额外鸟瞰图和去噪提升渲染质量，保持高速度优势
- 适合追求高质量街景重建的视觉算法研究者
神经辐射场（NeRF）和3D高斯溅射（3DGS）是两种主流的新视角合成方法，常表现出互补特性：3DGS渲染速度快，而NeRF渲染质量更高。受此启发，本文提出利用NeRF生成的图像增强3DGS。针对街景场景，我们采用预训练的特定街景NeRF方法生成目标3DGS的训练图像。在3DGS训练中，这些NeRF生成的图像用于消除街景输入中的瞬时物体，并生成鸟瞰视图作为额外视角，从而将NeRF的高质量渲染能力引入3DGS。我们进一步引入基于扩散模型的图像增强模块，以提升额外视角的图像质量。在1个合成数据集和2个真实数据集上的实验表明，所提方法在保持3DGS高速的同时，显著提升了街景渲染质量，且优于原始NeRF与3DGS的性能表现。
原文摘要 · Abstract (English)
Neural radiance field (NeRF) and 3D Gaussian splatting (3DGS) are two mainstream approaches for novel view synthesis. They often show complementary performance, i.e., 3DGS demonstrating faster rendering speed and NeRF demonstrating higher rendering quality. Motivated by this, we propose leveraging NeRF-rendered images for 3DGS. Specifically, we target street scenes and utilize a pre-trained street-specific NeRF method to produce training images for a target 3DGS method. In our 3DGS training, NeRF-rendered images are used to remove transient objects in street-level input views and to generate bird's-eye views as additional views, inheriting the higher-quality rendering of NeRF into 3DGS. We further incorporate a diffusion-based image enhancement to improve the image quality of the additional views. Experimental results on one synthetic and two real datasets demonstrate that our proposed method improves street-scene rendering while preserving the speed of 3DGS and the quality of NeRF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。