针对真实图像噪声相关性,设计新型盲区网络提升自监督去噪效果。
TM-BSN: Triangular-Masked Blind-Spot Network for Real-World Self-Supervised Image Denoising
- 用三角掩码卷积构建钻石形盲区,匹配相机噪声空间结构
- 无需下采样即可保留完整上下文,去噪性能超越现有方法
- 通过知识蒸馏融合多预测结果,兼顾精度与效率
盲区网络(BSNs)通过阻止模型访问目标像素,实现无需真值监督的自监督图像去噪。然而,该方法假设像素级噪声独立,这在真实sRGB图像中不成立,因摄像头信号处理(ISP)导致噪声存在空间相关性。现有方法虽通过下采样降低相关性,但改变了噪声统计并限制上下文利用。本文提出三角掩码盲区网络(TM-BSN),精准建模真实sRGB噪声的空间相关性。该相关性源于去马赛克过程,每个像素由邻近样本加权重建,权重随距离衰减,形成钻石形模式。为匹配此几何结构,我们引入三角掩码卷积,仅使用核的上三角区域,使盲区在原始分辨率下呈现钻石形状。该设计排除相关像素的同时充分利用无关上下文,无需下采样或后处理。此外,采用知识蒸馏将多个盲区预测的互补信息迁移至轻量U-Net,提升准确率与效率。在真实世界基准上的大量实验表明,本方法达到最先进性能,显著优于现有自监督方法。代码已开源。
原文摘要 · Abstract (English)
Blind-spot networks (BSNs) enable self-supervised image denoising by preventing access to the target pixel, allowing clean signal estimation without ground-truth supervision. However, this approach assumes pixel-wise noise independence, which is violated in real-world sRGB images due to spatially correlated noise from the camera's image signal processing (ISP) pipeline. While several methods employ downsampling to decorrelate noise, they alter noise statistics and limit the network's ability to utilize full contextual information. In this paper, we propose the Triangular-Masked Blind-Spot Network (TM-BSN), a novel blind-spot architecture that accurately models the spatial correlation of real sRGB noise. This correlation originates from demosaicing, where each pixel is reconstructed from neighboring samples with spatially decaying weights, resulting in a diamond-shaped pattern. To align the receptive field with this geometry, we introduce a triangular-masked convolution that restricts the kernel to its upper-triangular region, creating a diamond-shaped blind spot at the original resolution. This design excludes correlated pixels while fully leveraging uncorrelated context, eliminating the need for downsampling or post-processing. Furthermore, we use knowledge distillation to transfer complementary knowledge from multiple blind-spot predictions into a lightweight U-Net, improving both accuracy and efficiency. Extensive experiments on real-world benchmarks demonstrate that our method achieves state-of-the-art performance, significantly outperforming existing self-supervised approaches. Our code is available at https://github.com/parkjun210/TM-BSN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。