改进Mamba模型扫描方式,提升图像修复效果
MaIR: A Locality- and Continuity-Preserving Mamba for Image Restoration

- 采用S形扫描和条带区域保留图像局部连续性
- 通过通道注意力聚合不同展开序列,提升特征融合
- 在14个数据集上超越40个基线,适合图像修复任务
近期Mamba在图像修复中表现优异,但现有方法将2D图像按行列展平为多个独立的1D序列,分别处理后重组,忽略了自然图像中的局部关系与空间连续性,以及不同展开方式带来的差异。为此,本文提出MaIR模型,包含嵌套S形扫描策略(NSS)与序列洗牌注意力块(SSA)。NSS通过条带扫描区域和S形扫描路径,同时保持图像的局部性和空间连续性;SSA通过计算不同序列对应通道的注意力权重,实现序列间有效聚合。得益于上述设计,MaIR在14个挑战性数据集上的图像超分辨率、去噪、去模糊和去雾任务中均超越40个基线,达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
Recent advancements in Mamba have shown promising results in image restoration. These methods typically flatten 2D images into multiple distinct 1D sequences along rows and columns, process each sequence independently using selective scan operation, and recombine them to form the outputs. However, such a paradigm overlooks two vital aspects: i) the local relationships and spatial continuity inherent in natural images, and ii) the discrepancies among sequences unfolded through totally different ways. To overcome the drawbacks, we explore two problems in Mamba-based restoration methods: i) how to design a scanning strategy preserving both locality and continuity while facilitating restoration, and ii) how to aggregate the distinct sequences unfolded in totally different ways. To address these problems, we propose a novel Mamba-based Image Restoration model (MaIR), which consists of Nested S-shaped Scanning strategy (NSS) and Sequence Shuffle Attention block (SSA). Specifically, NSS preserves locality and continuity of the input images through the stripe-based scanning region and the S-shaped scanning path, respectively. SSA aggregates sequences through calculating attention weights within the corresponding channels of different sequences. Thanks to NSS and SSA, MaIR surpasses 40 baselines across 14 challenging datasets, achieving state-of-the-art performance on the tasks of image super-resolution, denoising, deblurring and dehazing. The code is available at https://github.com/XLearning-SCU/2025-CVPR-MaIR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。