通过多级注意力图网络,同时捕捉图像的局部与全局特征以提升修复效果。
Multi-level Attention-guided Graph Neural Network for Image Restoration
- 构建局部块图与元素图,用多注意力机制动态学习特征间关系。
- 在多个经典任务上达到当前最优性能,有效恢复缺失图像信息。
- 适合需要兼顾细节与整体结构的图像修复场景,如去噪、超分。
近年来,深度学习在图像修复领域取得显著进展。然而,大多数基于卷积神经网络的方法通常仅关注单一尺度,忽略了多尺度信息的融合。图像修复中,局部特征往往不足,需结合全局特征进行补充。尽管近期神经网络在特征提取方面进步明显,但许多模型未显式建模全局特征,也未考虑全局与局部特征间的关联。本文提出多级注意力引导的图神经网络,通过多注意力机制在特征图内显式构建元素块图和元素图,以提取局部结构特征与全局表示信息。由于网络在图像退化过程中难以有效提取全局信息,可利用局部特征块的结构信息来修正和补充全局信息;反之,当特征图中的元素块信息缺失时,也可借助全局元素表示信息进行精炼。网络中的图通过多注意力机制学习实时动态连接,并通过图卷积算法实现信息传播与聚合。结合局部元素块信息与全局元素表示信息,该算法能更有效地恢复图像中的缺失内容。在多个经典图像修复任务上的实验结果表明,所提方法具有优异性能,达到当前最优水平。
原文摘要 · Abstract (English)
In recent years, deep learning has achieved remarkable success in the field of image restoration. However, most convolutional neural network-based methods typically focus on a single scale, neglecting the incorporation of multi-scale information. In image restoration tasks, local features of an image are often insufficient, necessitating the integration of global features to complement them. Although recent neural network algorithms have made significant strides in feature extraction, many models do not explicitly model global features or consider the relationship between global and local features. This paper proposes multi-level attention-guided graph neural network. The proposed network explicitly constructs element block graphs and element graphs within feature maps using multi-attention mechanisms to extract both local structural features and global representation information of the image. Since the network struggles to effectively extract global information during image degradation, the structural information of local feature blocks can be used to correct and supplement the global information. Similarly, when element block information in the feature map is missing, it can be refined using global element representation information. The graph within the network learns real-time dynamic connections through the multi-attention mechanism, and information is propagated and aggregated via graph convolution algorithms. By combining local element block information and global element representation information from the feature map, the algorithm can more effectively restore missing information in the image. Experimental results on several classic image restoration tasks demonstrate the effectiveness of the proposed method, achieving state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。