提出时空盲区网络,提升自监督去噪视频的时序对齐与全局感知能力。
Spatiotemporal Blind-Spot Network with Calibrated Flow Alignment for Self-Supervised Video Denoising
- 设计双向盲区对齐块,增强长时序依赖建模能力
- 通过无监督光流蒸馏,降低噪声对运动估计的干扰
- 在合成与真实数据集上均实现领先性能,适合视频修复场景
自监督视频去噪旨在不依赖真实标签的情况下,利用视频自身信息恢复干净帧。现有方法常采用简单的特征堆叠或未充分分析的光流,导致帧间与帧内信息利用不充分,且忽视了自监督条件下光流对齐的潜力,造成结果偏差和去噪不足。为此,本文首次探索自监督设置下光流的可行性,提出时空盲区网络(STBN)以实现全局帧特征利用。在时间域,通过双向盲区特征传播块确保精确的时间对齐并有效捕捉长程依赖;在空间域,引入感受野扩展模块,增强感知范围。此外,为降低光流估计对噪声的敏感性,提出无监督光流蒸馏机制,在光流对齐中优化细粒度帧间交互。该方法在合成与真实视频去噪数据集上均表现优异。代码已公开于 https://github.com/ZKCCZ/STBN。
原文摘要 · Abstract (English)
Self-supervised video denoising aims to remove noise from videos without relying on ground truth data, leveraging the video itself to recover clean frames. Existing methods often rely on simplistic feature stacking or apply optical flow without thorough analysis. This results in suboptimal utilization of both inter-frame and intra-frame information, and it also neglects the potential of optical flow alignment under self-supervised conditions, leading to biased and insufficient denoising outcomes. To this end, we first explore the practicality of optical flow in the self-supervised setting and introduce a SpatioTemporal Blind-spot Network (STBN) for global frame feature utilization. In the temporal domain, we utilize bidirectional blind-spot feature propagation through the proposed blind-spot alignment block to ensure accurate temporal alignment and effectively capture long-range dependencies. In the spatial domain, we introduce the spatial receptive field expansion module, which enhances the receptive field and improves global perception capabilities. Additionally, to reduce the sensitivity of optical flow estimation to noise, we propose an unsupervised optical flow distillation mechanism that refines fine-grained inter-frame interactions during optical flow alignment. Our method demonstrates superior performance across both synthetic and real-world video denoising datasets. The source code is publicly available at https://github.com/ZKCCZ/STBN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。