用3D高斯点云加速闪光帧降噪,速度提升250倍且更保细节。
DenoiseGS: Gaussian Reconstruction Model for Burst Denoising
- 首次将3D高斯点云用于闪光帧降噪,提升效率。
- 在噪声条件下实现超越现有方法的图像质量与新视角生成能力。
- 提出自一致性与频域加权损失,有效保留纹理细节。
手持设备拍摄的闪光帧降噪对图像增强至关重要,但传统方法常受限于大运动或计算开销过高。本文提出DenoiseGS,首个利用3D高斯点云进行闪光帧降噪的框架。针对前向高斯重建模型在噪声输入下的点云退化和细节丢失问题,我们设计了高斯自一致性(GSC)损失,通过训练中同一模型生成的高质量点云来正则化噪声输入的几何预测,缓解潜在偏差与领域差异。同时引入对数加权频率(LWF)损失,在频域内自适应地放大高频差异权重,强化对细微结构的监督。大量实验表明,DenoiseGS在噪声条件下的闪光帧降噪与新视角生成任务中均显著超越基于NeRF的最先进方法,推理速度提升250倍。代码与模型已开源。
原文摘要 · Abstract (English)
Burst denoising methods are crucial for enhancing images captured on handheld devices, but they often struggle with large motion or suffer from prohibitive computational costs. In this paper, we propose DenoiseGS, the first framework to leverage the efficiency of 3D Gaussian Splatting for burst denoising. Our approach addresses two key challenges when applying feedforward Gaussian reconsturction model to noisy inputs: the degradation of Gaussian point clouds and the loss of fine details. To this end, we propose a Gaussian self-consistency (GSC) loss, which regularizes the geometry predicted from noisy inputs with high-quality Gaussian point clouds. These point clouds are generated from clean inputs by the same model that we are training, thereby alleviating potential bias or domain gaps. Additionally, we introduce a log-weighted frequency (LWF) loss to strengthen supervision within the spectral domain, effectively preserving fine-grained details. The LWF loss adaptively weights frequency discrepancies in a logarithmic manner, emphasizing challenging high-frequency details. Extensive experiments demonstrate that DenoiseGS significantly exceeds the state-of-the-art NeRF-based methods on both burst denoising and novel view synthesis under noisy conditions, while achieving 250$\times$ faster inference speed. Code and models are released at https://github.com/yscheng04/DenoiseGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。