提出新型卷积网络CCNet,提升图像修复的上下文感知与高维特征表达能力。
Towards Context-aware Convolutional Network for Image Restoration
- 设计星操作残差模块(ERSM),通过上下文感知映射增强非线性特征表达
- 引入大动态融合模块(LDIM),实现超大感受野,高效捕捉长程上下文信息
- 在去雾、去模糊、去雪等任务中表现优于现有方法,模型复杂度低
图像修复(IR)旨在从退化观测中恢复高质量图像。尽管基于Transformer和注意力机制的卷积神经网络(CNN)在多个IR任务上取得进展,但现有卷积残差模块仍受限于难以映射到高维非线性特征空间,且局部感受野难以捕捉长距离上下文信息。同时,基于CNN的注意力模块或参数冗余,或感受野有限。为此,本文提出高效残差星模块(ERSM),通过上下文感知的“星操作”(逐元素乘法)将特征映射至极高维非线性空间,显著增强表示学习能力。为进一步提升上下文信息提取,提出大动态融合模块(LDIM),具备极大连贯感受野,可动态高效整合更多上下文信息,显著提升重建性能。将ERSM与LDIM集成至U型主干网络,构建具有强大上下文高维映射与丰富上下文信息建模能力的上下文感知卷积网络(CCNet)。大量实验表明,该方法以较低模型复杂度,在图像去雾、运动去模糊、去雪等多个任务中均优于当前最优方法。
原文摘要 · Abstract (English)
Image restoration (IR) is a long-standing task to recover a high-quality image from its corrupted observation. Recently, transformer-based algorithms and some attention-based convolutional neural networks (CNNs) have presented promising results on several IR tasks. However, existing convolutional residual building modules for IR encounter limited ability to map inputs into high-dimensional and non-linear feature spaces, and their local receptive fields have difficulty in capturing long-range context information like Transformer. Besides, CNN-based attention modules for IR either face static abundant parameters or have limited receptive fields. To address the first issue, we propose an efficient residual star module (ERSM) that includes context-aware "star operation" (element-wise multiplication) to contextually map features into exceedingly high-dimensional and non-linear feature spaces, which greatly enhances representation learning. To further boost the extraction of contextual information, as for the second issue, we propose a large dynamic integration module (LDIM) which possesses an extremely large receptive field. Thus, LDIM can dynamically and efficiently integrate more contextual information that helps to further significantly improve the reconstruction performance. Integrating ERSM and LDIM into an U-shaped backbone, we propose a context-aware convolutional network (CCNet) with powerful learning ability for contextual high-dimensional mapping and abundant contextual information. Extensive experiments show that our CCNet with low model complexity achieves superior performance compared to other state-of-the-art IR methods on several IR tasks, including image dehazing, image motion deblurring, and image desnowing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。