arXiv:2410.02572eess.IVcs.CV2024-10被引 3

提出联合前后去马赛克阶段降噪的视频降噪方法,适应不同噪声水平。

Combining Pre- and Post-Demosaicking Noise Removal for RAW Video

  • 基于自相似性加权融合前后去马赛克降噪器
  • 高噪声下前置降噪影响更大,提升图像质量
  • 无需重训练,可自适应任意噪声水平,适合实际拍摄

降噪是将相机传感器捕获的数据转换为可显示图像或视频的核心步骤。传统上在去马赛克前进行,但近年研究也探索了顺序调换甚至联合处理。深度学习虽显著提升了降噪质量,但现有神经网络对新噪声水平和场景适应能力仍弱,难以满足真实应用需求。本文提出一种基于自相似性的降噪方案,对拜耳阵列色彩滤波阵列(CFA)视频数据的预去马赛克与后去马赛克降噪器进行加权融合。实验表明,合理平衡两者可提升图像质量;且在高噪声条件下,前置降噪的权重应更高。同时,在每个降噪器前引入时间轨迹预过滤,进一步改善纹理重建效果。该方法仅需估计传感器噪声模型,能准确适应任意噪声水平,性能媲美当前最优方法,适用于真实视频拍摄场景。

原文摘要 · Abstract (English)

Denoising is one of the fundamental steps of the processing pipeline that converts data captured by a camera sensor into a display-ready image or video. It is generally performed early in the pipeline, usually before demosaicking, although studies swapping their order or even conducting them jointly have been proposed. With the advent of deep learning, the quality of denoising algorithms has steadily increased. Even so, modern neural networks still have a hard time adapting to new noise levels and scenes, which is indispensable for real-world applications. With those in mind, we propose a self-similarity-based denoising scheme that weights both a pre- and a post-demosaicking denoiser for Bayer-patterned CFA video data. We show that a balance between the two leads to better image quality, and we empirically find that higher noise levels benefit from a higher influence pre-demosaicking. We also integrate temporal trajectory prefiltering steps before each denoiser, which further improve texture reconstruction. The proposed method only requires an estimation of the noise model at the sensor, accurately adapts to any noise level, and is competitive with the state of the art, making it suitable for real-world videography.

视频降噪去马赛克自相似性深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。