融合动态注意力与Transformer的图像修复新架构,兼顾效果与效率。
Joint multi-dimensional dynamic attention and transformer for general image restoration
- 用多维动态注意力增强卷积,捕捉复杂退化特征。
- 在五个任务上均优于现有方法,且计算开销更低。
- 适合图像修复、视觉理解等需要高效高质恢复的场景。
户外图像常受雨、雾、噪声等多重退化影响,降低画质并挑战高层视觉任务。现有修复方法难以在复杂退化下保持高效。本文提出一种结合多维动态注意力与自注意力的U-Net架构:编码器-解码器使用纯CNN,潜空间引入纯Transformer;设计具备多维动态注意力的卷积核以高效建模多样退化输入;通过转置自注意力块增强全局特征提取能力。大量实验表明,该方法在去雨、去模糊、去噪、去雾及图像增强五项任务中均实现性能与计算复杂度的更好平衡,并显著提升高层视觉任务表现。代码将开源于 https://github.com/House-yuyu/MDDA-former。
原文摘要 · Abstract (English)
Outdoor images often suffer from severe degradation due to rain, haze, and noise, impairing image quality and challenging high-level tasks. Current image restoration methods struggle to handle complex degradation while maintaining efficiency. This paper introduces a novel image restoration architecture that combines multi-dimensional dynamic attention and self-attention within a U-Net framework. To leverage the global modeling capabilities of transformers and the local modeling capabilities of convolutions, we integrate sole CNNs in the encoder-decoder and sole transformers in the latent layer. Additionally, we design convolutional kernels with selected multi-dimensional dynamic attention to capture diverse degraded inputs efficiently. A transformer block with transposed self-attention further enhances global feature extraction while maintaining efficiency. Extensive experiments demonstrate that our method achieves a better balance between performance and computational complexity across five image restoration tasks: deraining, deblurring, denoising, dehazing, and enhancement, as well as superior performance for high-level vision tasks. The source code will be available at https://github.com/House-yuyu/MDDA-former.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。