用隐空间LSTM建模运动，实现轻量级盲视频去噪
LLVD: LSTM-based Explicit Motion Modeling in Latent Space for Blind Video Denoising
- 在编码特征域用LSTM捕捉时序运动，保持画面连续性
- 相比当前最优方法，RAW去噪提升0.3dB，计算量减少59%
- 无需噪声先验知识，适合真实场景的通用去噪
视频修复在恢复因拍摄（传感器噪声、运动模糊等）、保存/分享（压缩、缩放等）及编辑引入的退化内容方面起着关键作用。本文提出一种针对拍摄阶段引入噪声的新算法，旨在通过降低不必要的噪声伪影来提升视频视觉质量。我们设计了基于隐空间的长短期记忆视频去噪模型（LLVD），一个端到端的盲去噪模型。LLVD创新性地结合空间与时间特征提取，将长短期记忆（LSTM）层嵌入编码特征域，有效维持恢复视频的连贯性并减少闪烁。此外，在编码特征域处理帧显著降低计算开销，形成极轻量级架构。其盲去噪特性使其适用于缺乏噪声先验信息的真实复杂场景。实验表明，LLVD在合成噪声和实际捕获噪声上均表现优异：在RAW去噪上优于当前最先进方法0.3dB，同时计算复杂度降低59%。
原文摘要 · Abstract (English)
Video restoration plays a pivotal role in revitalizing degraded video content by rectifying imperfections caused by various degradations introduced during capturing (sensor noise, motion blur, etc.), saving/sharing (compression, resizing, etc.) and editing. This paper introduces a novel algorithm designed for scenarios where noise is introduced during video capture, aiming to enhance the visual quality of videos by reducing unwanted noise artifacts. We propose the Latent space LSTM Video Denoiser (LLVD), an end-to-end blind denoising model. LLVD uniquely combines spatial and temporal feature extraction, employing Long Short Term Memory (LSTM) within the encoded feature domain. This integration of LSTM layers is crucial for maintaining continuity and minimizing flicker in the restored video. Moreover, processing frames in the encoded feature domain significantly reduces computations, resulting in a very lightweight architecture. LLVD's blind nature makes it versatile for real, in-the-wild denoising scenarios where prior information about noise characteristics is not available. Experiments reveal that LLVD demonstrates excellent performance for both synthetic and captured noise. Specifically, LLVD surpasses the current State-Of-The-Art (SOTA) in RAW denoising by 0.3dB, while also achieving a 59\% reduction in computational complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。