提出可变形滑窗注意力,提升图像修复的特征融合与感受野。
DSwinIR: Rethinking Window-based Attention for Image Restoration
- 用基于令牌的滑动窗口替代固定网格分区,减少边界伪影。
- 引入内容感知偏移,动态调整感受野,聚焦重要图像区域。
- 在多任务修复上超越现有模型,最高提升0.87 dB。
图像修复因深度学习模型的发展取得了显著进展。基于Transformer的模型,尤其是采用窗口自注意力机制的,已成为主流。然而,其性能受限于刚性、非重叠的窗口划分方式,导致窗口间特征交互不足且感受野有限。为此,本文提出用于图像修复的可变形滑动窗口Transformer(DSwinIR),引入一种新的注意力机制——可变形滑动窗口(DSwin)注意力。该机制采用以令牌为中心、内容感知的范式,突破传统网格与固定窗口的限制。包含两个互补组件:首先,以令牌为中心的滑动窗口机制有效消除边界伪影;其次,引入内容感知的可变形采样策略,使注意力能学习数据相关的偏移量,主动塑造感受野以聚焦最信息丰富的图像区域。大量实验表明,DSwinIR表现优异,在多个基准上达到领先水平。例如,在全任务图像修复中,相比最新骨干模型GridFormer,DSwinIR在三任务基准上提升0.53 dB,五任务基准上提升0.87 dB。
原文摘要 · Abstract (English)
Image restoration has witnessed significant advancements with the development of deep learning models. Transformer-based models, particularly those using window-based self-attention, have become a dominant force. However, their performance is constrained by the rigid, non-overlapping window partitioning scheme, which leads to \textit{insufficient feature interaction across windows and limited receptive fields}. This highlights the need for more adaptive and flexible attention mechanisms. In this paper, we propose the Deformable Sliding Window Transformer for Image Restoration (DSwinIR), a new attention mechanism: the {Deformable Sliding Window (DSwin) Attention}. {This mechanism introduces a token-centric and content-aware paradigm that moves beyond the grid and fixed window partition.} It comprises two complementary components. First, it replaces the rigid partitioning with a \textit{token-centric sliding window} paradigm, {making it effective at eliminating boundary artifacts}. Second, it incorporates a \textit{content-aware deformable sampling} strategy, which allows the attention mechanism to learn data-dependent offsets and actively shape its receptive field to focus on the most informative image regions. Extensive experiments show that DSwinIR achieves strong results, including state-of-the-art performance on several evaluated benchmarks. For instance, in all-in-one image restoration, our DSwinIR surpasses the most recent backbone GridFormer by 0.53 dB on the three-task benchmark and 0.87 dB on the five-task benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。