DualDn通过双域联合去噪,提升相机图像去噪的通用性。
DualDn: Dual-domain Denoising via Differentiable ISP
- 设计双网络结构:分别在原始域和sRGB域进行去噪,适应不同噪声特性。
- 端到端训练可适配未见噪声、ISP参数及新流水线,性能超越现有方法。
- 无需重训即可作为即插即用模块,实测优于商用相机原生去噪。
图像去噪是相机图像信号处理(ISP)流水线中的关键环节。传统方法通常在原始数据域(raw domain)或ISP输出的sRGB图像域(sRGB domain)中应用去噪器,但两者均有局限:原始域残留噪声可能被后续ISP放大;sRGB域难以处理空间变化的噪声,因仅能观察到经ISP扭曲后的噪声。因此,现有方法多局限于特定噪声分布与ISP配置。为此,本文提出DualDn——一种基于学习的双域去噪方法。它包含两个去噪网络:一个在原始域处理传感器特异性噪声与空间变噪声水平,另一个在sRGB域应对ISP差异并消除被放大的残余噪声。两网络通过可微分的ISP连接,实现端到端训练,并在推理阶段丢弃。该设计使DualDn具备更强泛化能力,可适应不同未知噪声、ISP参数甚至全新ISP流水线。实验表明,DualDn达到当前最优性能,且可灵活适配不同去噪架构。此外,其作为即插即用模块部署于真实相机时,无需重训仍优于商业相机内置去噪效果。
原文摘要 · Abstract (English)
Image denoising is a critical component in a camera's Image Signal Processing (ISP) pipeline. There are two typical ways to inject a denoiser into the ISP pipeline: applying a denoiser directly to captured raw frames (raw domain) or to the ISP's output sRGB images (sRGB domain). However, both approaches have their limitations. Residual noise from raw-domain denoising can be amplified by the subsequent ISP processing, and the sRGB domain struggles to handle spatially varying noise since it only sees noise distorted by the ISP. Consequently, most raw or sRGB domain denoising works only for specific noise distributions and ISP configurations. To address these challenges, we propose DualDn, a novel learning-based dual-domain denoising. Unlike previous single-domain denoising, DualDn consists of two denoising networks: one in the raw domain and one in the sRGB domain. The raw domain denoising adapts to sensor-specific noise as well as spatially varying noise levels, while the sRGB domain denoising adapts to ISP variations and removes residual noise amplified by the ISP. Both denoising networks are connected with a differentiable ISP, which is trained end-to-end and discarded during the inference stage. With this design, DualDn achieves greater generalizability compared to most learning-based denoising methods, as it can adapt to different unseen noises, ISP parameters, and even novel ISP pipelines. Experiments show that DualDn achieves state-of-the-art performance and can adapt to different denoising architectures. Moreover, DualDn can be used as a plug-and-play denoising module with real cameras without retraining, and still demonstrate better performance than commercial on-camera denoising. The project website is available at: https://openimaginglab.github.io/DualDn/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。