arXiv:2603.02581cs.CV2026-03TPAMI被引 2

用可学习词典实现全局依赖建模,提升图像修复效率与效果

ATD: Improved Transformer with Adaptive Token Dictionary for Image Restoration

论文配图:ATD: Improved Transformer with Adaptive Token Dictionary for Image Restoration
图 1 · 摘自论文原文
  • 引入可学习词典捕捉图像结构先验,通过交叉注意力增强特征
  • 线性复杂度下实现全局感受野,超分辨率任务上达到新最好性能
  • 适用于去噪、压缩伪影去除等多类图像修复任务,适合实际部署

近期,Transformer 在图像超分辨率、去噪等图像修复任务中表现优异,但性能与计算开销难以平衡。由于自注意力的二次复杂度,现有方法通常限制注意力范围在局部窗口,导致感受野受限。为此,我们提出自适应词典(ATD),一种新型Transformer架构,可在线性复杂度下建模全局依赖。该模型引入可学习的词典,训练时总结外部图像先验(典型图像结构)。通过词典交叉注意力(TDCA)机制,输入特征与词典交互以增强表达。此外,利用TDCA注意力图中的类别信息,将输入特征分组为多个类别簇,每类作为注意力分组;并将类别信息融入前馈网络以提升特征融合。ATD及其轻量版ATD-light在多个超分辨率基准上达到当前最优。我们还构建了多尺度变体ATD-U,用于去噪和JPEG压缩伪影去除。大量实验表明,所提模型在定量与定性指标上均具优势。

原文摘要 · Abstract (English)

Recently, Transformers have gained significant popularity in image restoration tasks such as image super-resolution and denoising, owing to their superior performance. However, balancing performance and computational burden remains a long-standing problem for transformer-based architectures. Due to the quadratic complexity of self-attention, existing methods often restrict attention to local windows, resulting in limited receptive field and suboptimal performance. To address this issue, we propose Adaptive Token Dictionary (ATD), a novel transformer-based architecture for image restoration that enables global dependency modeling with linear complexity relative to image size. The ATD model incorporates a learnable token dictionary, which summarizes external image priors (i.e., typical image structures) during the training process. To utilize this information, we introduce a token dictionary cross-attention (TDCA) mechanism that enhances the input features via interaction with the learned dictionary. Furthermore, we exploit the category information embedded in the TDCA attention maps to group input features into multiple categories, each representing a cluster of similar features across the image and serving as an attention group. We also integrate the learned category information into the feed-forward network to further improve feature fusion. ATD and its lightweight version ATD-light, achieve state-of-the-art performance on multiple image super-resolution benchmarks. Moreover, we develop ATD-U, a multi-scale variant of ATD, to address other image restoration tasks, including image denoising and JPEG compression artifacts removal. Extensive experiments demonstrate the superiority of out proposed models, both quantitatively and qualitatively.

图像修复Transformer词典学习超分辨率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。