单帧延迟实现顶级去噪，适合实时视频处理。
A Practical Gated Recurrent Transformer Network Incorporating Multiple Fusions for Video Denoising
- 用门控循环结构融合前后帧信息，仅需单帧延迟。
- 在SOTA性能下实现单帧延迟，比现有方法快16倍。
- 适合实时相机、低延迟视频系统应用。
当前顶尖的视频去噪方法采用多帧同时处理机制，导致显著延迟（如16帧），不适用于实时摄像头。为克服这一限制，我们提出一种多融合门控递归Transformer网络（GRTN），在仅单帧延迟下达到顶尖去噪性能。具体而言，空间去噪模块提取当前帧特征，重置门从前一帧选择相关信息，并通过时间去噪模块与当前帧特征融合；更新门进一步将结果与前一帧特征混合，重建模块则将其与当前帧整合。为在噪声特征上稳健计算注意力，我们在空间和时间去噪模块中引入残差简化Swin Transformer与欧氏距离（RSSTE）。客观与主观对比结果表明，我们的GRTN在仅单帧延迟下实现了与顶尖多帧延迟网络相当的去噪效果。
原文摘要 · Abstract (English)
State-of-the-art (SOTA) video denoising methods employ multi-frame simultaneous denoising mechanisms, resulting in significant delays (e.g., 16 frames), making them impractical for real-time cameras. To overcome this limitation, we propose a multi-fusion gated recurrent Transformer network (GRTN) that achieves SOTA denoising performance with only a single-frame delay. Specifically, the spatial denoising module extracts features from the current frame, while the reset gate selects relevant information from the previous frame and fuses it with current frame features via the temporal denoising module. The update gate then further blends this result with the previous frame features, and the reconstruction module integrates it with the current frame. To robustly compute attention for noisy features, we propose a residual simplified Swin Transformer with Euclidean distance (RSSTE) in the spatial and temporal denoising modules. Comparative objective and subjective results show that our GRTN achieves denoising performance comparable to SOTA multi-frame delay networks, with only a single-frame delay.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。