arXiv:2505.05504eess.IVcs.CV2025-05TPAMI被引 3

用多领域学习提升图像修复,兼顾效果与效率。

Image Restoration via Multi-domain Learning

  • 设计跨域感知的Transformer结构,融合空间、小波、傅里叶多尺度特征。
  • 在10项修复任务中超越主流方法,参数量小、推理快、性能优。
  • 适合需要轻量化高精度修复的应用,如移动端实时图像增强。

由于恶劣大气和成像条件,自然图像常出现多种退化现象。尽管近期Transformer架构在各类修复任务中表现优异,但其模型复杂度高,给训练和实时部署带来挑战。现有方法多聚焦于特定退化,忽视不同退化间的共性。本文从多领域视角分析退化现象,挖掘共性先验。提出新型修复框架,将多领域学习融入Transformer:在Token Mixer中引入空间-小波-傅里叶多域结构,实现局部-区域-全局多感受野建模,替代原始自注意力;在前馈网络中加入多尺度学习,融合不同分辨率的多域特征。在包括去雾、去雪、运动模糊去除、散焦模糊去除、雨线/雨滴去除、云层去除、阴影去除、水下增强和低光增强在内的十项任务上,实验结果表明,所提模型优于当前最优方法,在修复性能、参数量、计算成本和推理延迟间取得良好平衡。

原文摘要 · Abstract (English)

Due to adverse atmospheric and imaging conditions, natural images suffer from various degradation phenomena. Consequently, image restoration has emerged as a key solution and garnered substantial attention. Although recent Transformer architectures have demonstrated impressive success across various restoration tasks, their considerable model complexity poses significant challenges for both training and real-time deployment. Furthermore, instead of investigating the commonalities among different degradations, most existing restoration methods focus on modifying Transformer under limited restoration priors. In this work, we first review various degradation phenomena under multi-domain perspective, identifying common priors. Then, we introduce a novel restoration framework, which integrates multi-domain learning into Transformer. Specifically, in Token Mixer, we propose a Spatial-Wavelet-Fourier multi-domain structure that facilitates local-region-global multi-receptive field modeling to replace vanilla self-attention. Additionally, in Feed-Forward Network, we incorporate multi-scale learning to fuse multi-domain features at different resolutions. Comprehensive experimental results across ten restoration tasks, such as dehazing, desnowing, motion deblurring, defocus deblurring, rain streak/raindrop removal, cloud removal, shadow removal, underwater enhancement and low-light enhancement, demonstrate that our proposed model outperforms state-of-the-art methods and achieves a favorable trade-off among restoration performance, parameter size, computational cost and inference latency. The code is available at: https://github.com/deng-ai-lab/SWFormer.

图像修复Transformer多域学习轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。