用变分残差特征提升3D高分辨率图像合成质量
SuperGS: Super-Resolution 3D Gaussian Splatting Enhanced by Variational Residual Features and Uncertainty-Augmented Learning
- 两阶段渐进训练，低分辨率特征场初始化并引导超分优化
- 通过特征方差估计不确定性，指导细节增强与点云优化
- 多视角联合学习缓解伪标签歧义，适合真实场景高清重建
近期，3D高斯点绘（3DGS）在新视图合成（NVS）中表现出色，具备实时渲染与高质量输出能力。然而，由于输入视图分辨率较低，其在高分辨率新视图合成（HRNVS）任务中面临挑战。为此，本文提出超分辨率3DGS（SuperGS），采用两阶段粗到精训练框架。该框架利用潜在特征场表示低分辨率场景，作为超分辨率优化的初始状态与基础信息。引入变分残差特征以增强高分辨率细节，并利用其方差作为不确定性估计，指导点云密化过程与损失计算。此外，多视角联合学习策略有效缓解了伪标签因多视角不一致带来的歧义问题。大量实验表明，SuperGS仅使用低分辨率输入，在真实世界与合成数据集上均优于现有先进方法。代码已开源。
原文摘要 · Abstract (English)
Recently, 3D Gaussian Splatting (3DGS) has exceled in novel view synthesis (NVS) with its real-time rendering capabilities and superior quality. However, it faces challenges for high-resolution novel view synthesis (HRNVS) due to the coarse nature of primitives derived from low-resolution input views. To address this issue, we propose Super-Resolution 3DGS (SuperGS), which is an expansion of 3DGS designed with a two-stage coarse-to-fine training framework. In this framework, we use a latent feature field to represent the low-resolution scene, serving as both the initialization and foundational information for super-resolution optimization. Additionally, we introduce variational residual features to enhance high-resolution details, using their variance as uncertainty estimates to guide the densification process and loss computation. Furthermore, the introduction of a multi-view joint learning approach helps mitigate ambiguities caused by multi-view inconsistencies in the pseudo labels. Extensive experiments demonstrate that SuperGS surpasses state-of-the-art HRNVS methods on both real-world and synthetic datasets using only low-resolution inputs. Code is available at https://github.com/SYXieee/SuperGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。