解决3D高斯溅射在异常视角下的视觉噪声问题。
Extreme Views: 3DGS Filter for Novel View Synthesis from Out-of-Distribution Camera Poses
- 基于中间梯度敏感度设计实时渲染感知滤波器。
- 显著提升异常视角下的图像质量与一致性。
- 无需重训练，可直接嵌入现有渲染流程。
当从显著偏离训练数据分布的相机位置观察3D高斯溅射（3DGS）模型时，常出现明显视觉噪声。这些伪影源于模型在这些外推区域缺乏训练数据，导致密度、颜色和几何预测不确定性。为此，我们提出一种新型实时渲染感知滤波方法。该方法利用中间梯度导出的敏感度分数，明确针对各向异性方向引起的不稳定性，而非各向同性方差。该滤波机制直接缓解生成不确定性，使3D重建系统在用户自由导航至原始训练视角之外时仍保持高视觉保真度。实验表明，相较于基于NeRF的方法如BayesRays，本方法显著提升视觉质量、真实感与一致性。关键优势在于：滤波器可无缝集成于现有3DGS渲染管线，实现实时运行，无需耗时的后期重训练或微调。代码与结果见 https://damian-bowness.github.io/EV3DGS。
原文摘要 · Abstract (English)
When viewing a 3D Gaussian Splatting (3DGS) model from camera positions significantly outside the training data distribution, substantial visual noise commonly occurs. These artifacts result from the lack of training data in these extrapolated regions, leading to uncertain density, color, and geometry predictions from the model. To address this issue, we propose a novel real-time render-aware filtering method. Our approach leverages sensitivity scores derived from intermediate gradients, explicitly targeting instabilities caused by anisotropic orientations rather than isotropic variance. This filtering method directly addresses the core issue of generative uncertainty, allowing 3D reconstruction systems to maintain high visual fidelity even when users freely navigate outside the original training viewpoints. Experimental evaluation demonstrates that our method substantially improves visual quality, realism, and consistency compared to existing Neural Radiance Field (NeRF)-based approaches such as BayesRays. Critically, our filter seamlessly integrates into existing 3DGS rendering pipelines in real-time, unlike methods that require extensive post-hoc retraining or fine-tuning. Code and results at https://damian-bowness.github.io/EV3DGS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。