针对屏下摄像头成像模糊问题,提出自适应恢复方法,提升细节还原能力。
UCMNet: Uncertainty-Aware Context Memory Network for Under-Display Camera Image Restoration
- 基于不确定性感知的上下文记忆网络,动态调整修复策略。
- 在多个数据集上超越现有方法,参数量减少30%。
- 适合手机图像增强与低光照复杂退化场景应用。
屏下摄像头(UDCs)通过将传感器置于屏幕下方实现全面屏设计,但显示层导致的光衍射和散射会造成空间变化且复杂的退化,显著损失高频细节。现有基于点扩散函数(PSF)的物理建模方法和频域分离网络虽能有效重建低频结构并保持整体色彩一致性,但在处理复杂空间变化退化时仍难以恢复精细细节。为此,本文提出轻量级不确定性感知上下文记忆网络(UCMNet),用于屏下摄像头图像恢复。不同于以往采用统一修复的方法,UCMNet通过不确定性驱动损失学习退化区域的不确定性图,量化由衍射和散射引起的局部不确定性,并引导记忆库从上下文库中检索区域自适应的上下文信息,从而有效建模屏下成像固有的非均匀退化特性。借助该不确定性先验,UCMNet在多个基准测试中达到当前最优性能,且参数量比之前模型减少30%。
原文摘要 · Abstract (English)
Under-display cameras (UDCs) allow for full-screen designs by positioning the imaging sensor underneath the display. Nonetheless, light diffraction and scattering through the various display layers result in spatially varying and complex degradations, which significantly reduce high-frequency details. Current PSF-based physical modeling techniques and frequency-separation networks are effective at reconstructing low-frequency structures and maintaining overall color consistency. However, they still face challenges in recovering fine details when dealing with complex, spatially varying degradation. To solve this problem, we propose a lightweight \textbf{U}ncertainty-aware \textbf{C}ontext-\textbf{M}emory \textbf{Network} (\textbf{UCMNet}), for UDC image restoration. Unlike previous methods that apply uniform restoration, UCMNet performs uncertainty-aware adaptive processing to restore high-frequency details in regions with varying degradations. The estimated uncertainty maps, learned through an uncertainty-driven loss, quantify spatial uncertainty induced by diffraction and scattering, and guide the Memory Bank to retrieve region-adaptive context from the Context Bank. This process enables effective modeling of the non-uniform degradation characteristics inherent to UDC imaging. Leveraging this uncertainty as a prior, UCMNet achieves state-of-the-art performance on multiple benchmarks with 30\% fewer parameters than previous models. Project page: \href{https://kdhrick2222.github.io/projects/UCMNet/}{https://kdhrick2222.github.io/projects/UCMNet}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。