arXiv:2409.11256cs.CVeess.IV2024-09ECCV被引 13

用预训练图像去噪器+时间模块,无监督实现视频去噪

Temporal As a Plugin: Unsupervised Video Denoising with Pre-Trained Image Denoisers

论文配图:Temporal As a Plugin: Unsupervised Video Denoising with Pre-Trained Image Denoisers
图 1 · 摘自论文原文
  • 将预训练图像去噪器与可调时间模块结合,利用时间信息增强去噪
  • 在sRGB和raw数据集上优于现有无监督方法,峰值性能提升显著
  • 适合缺乏成对视频数据的场景,可快速部署于真实视频去噪任务

深度学习在图像和视频去噪方面取得了显著进展,但动态场景中难以获取成对的带噪与无噪视频数据,限制了视频去噪技术的实际应用。相比之下,图像去噪的成对数据更易获取,因此预训练的图像去噪器可作为可靠的空域先验。本文提出一种新颖的无监督视频去噪框架「Temporal As a Plugin」(TAP),将可调节的时间模块嵌入预训练图像去噪器中,通过跨帧时间信息互补其空间去噪能力。此外,我们设计了一种渐进式微调策略,利用生成的伪干净视频帧逐步优化每个时间模块,持续提升网络性能。在sRGB和raw视频去噪数据集上的实验表明,该框架优于其他无监督方法。

原文摘要 · Abstract (English)

Recent advancements in deep learning have shown impressive results in image and video denoising, leveraging extensive pairs of noisy and noise-free data for supervision. However, the challenge of acquiring paired videos for dynamic scenes hampers the practical deployment of deep video denoising techniques. In contrast, this obstacle is less pronounced in image denoising, where paired data is more readily available. Thus, a well-trained image denoiser could serve as a reliable spatial prior for video denoising. In this paper, we propose a novel unsupervised video denoising framework, named ``Temporal As a Plugin'' (TAP), which integrates tunable temporal modules into a pre-trained image denoiser. By incorporating temporal modules, our method can harness temporal information across noisy frames, complementing its power of spatial denoising. Furthermore, we introduce a progressive fine-tuning strategy that refines each temporal module using the generated pseudo clean video frames, progressively enhancing the network's denoising performance. Compared to other unsupervised video denoising methods, our framework demonstrates superior performance on both sRGB and raw video denoising datasets.

视频去噪无监督学习图像去噪时间模块

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。