融合空间与频域信息,提升屏下摄像头图像恢复效果
Integrating Spatial and Frequency Information for Under-Display Camera Image Restoration
- 构建多级网络SFIM,分别处理局部细节与全局纹理
- 在三个基准上优于现有方法,尤其对杂散光抑制更优
- 适合从事图像修复、移动设备成像的开发者参考
屏下摄像头(UDC)将摄像头置于屏幕下方,导致图像出现噪声、模糊、透光率下降和杂散光等复杂退化。尽管已有研究主要聚焦于空间域的衍射消除,但较少探索频率域的潜力。本文重新审视傅里叶空间中的退化特性,发现杂散光存在固有的频率先验。基于此,提出新型多级深度神经网络SFIM,通过空间域块(SDB)捕捉局部纹理如噪声和模糊,频率域块(FDB)处理大范围不规则纹理损失如杂散光,再经注意力融合块(AMIB)实现跨域交互。实验在三个UDC基准上验证了SFIM的优越性,定量与定性评估均表现领先。
原文摘要 · Abstract (English)
Under-Display Camera (UDC) houses a digital camera lens under a display panel. However, UDC introduces complex degradations such as noise, blur, decrease in transmittance, and flare. Despite the remarkable progress, previous research on UDC mainly focuses on eliminating diffraction in the spatial domain and rarely explores its potential in the frequency domain. It is essential to consider both the spatial and frequency domains effectively. For example, degradations, such as noise and blur, can be addressed by local information (e.g., CNN kernels in the spatial domain). At the same time, tackling flares may require leveraging global information (e.g., the frequency domain). In this paper, we revisit the UDC degradations in the Fourier space and figure out intrinsic frequency priors that imply the presence of the flares. Based on this observation, we propose a novel multi-level DNN architecture called SFIM. It efficiently restores UDC-distorted images by integrating local and global (the collective contribution of all points in the image) information. The architecture exploits CNNs to capture local information and FFT-based models to capture global information. SFIM comprises a spatial domain block (SDB), a Frequency Domain Block (FDB), and an Attention-based Multi-level Integration Block (AMIB). Specifically, SDB focuses more on detailed textures such as noise and blur, FDB emphasizes irregular texture loss in extensive areas such as flare, and AMIB enables effective cross-domain interaction. SFIM's superior performance over state-of-the-art approaches is demonstrated through rigorous quantitative and qualitative assessments across three UDC benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。