arXiv:2501.16583cs.CV2025-01IJCAI被引 41

让Mamba模型关注复杂纹理区域,提升图像修复效率与质量

Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restoration

  • 通过调节状态空间矩阵,让模型聚焦复杂纹理区域
  • 在超分辨率、去雨、低光增强任务中均达顶尖性能
  • 适合追求高效高质图像修复的开发者和研究者

图像修复旨在恢复退化图像的细节并增强对比度。随着对高质量成像(如4K和8K)的需求增长,如何在修复质量与计算效率之间取得平衡变得愈发关键。现有基于CNN、Transformer或其混合方法的模型对图像全局采用统一深度表征,难以有效建模长程依赖,且忽略退化区域的空间特性(纹理越丰富,损伤越严重),导致难以兼顾性能与效率。为此,我们提出一种新型纹理感知图像修复方法TAMambaIR,同时感知图像纹理并实现性能与效率的平衡。具体地,引入一种新颖的纹理感知状态空间模型,通过调制状态空间方程中的转移矩阵,增强纹理感知能力并提升效率;此外,设计多方向感知模块,在保持低计算开销的同时扩大多方向感受野。在图像超分辨率、去雨和低光增强等多个基准测试上,TAMambaIR均达到当前最优性能,并显著提升效率,验证了其作为高效鲁棒图像修复框架的有效性。

原文摘要 · Abstract (English)

Image restoration aims to recover details and enhance contrast in degraded images. With the growing demand for high-quality imaging (\textit{e.g.}, 4K and 8K), achieving a balance between restoration quality and computational efficiency has become increasingly critical. Existing methods, primarily based on CNNs, Transformers, or their hybrid approaches, apply uniform deep representation extraction across the image. However, these methods often struggle to effectively model long-range dependencies and largely overlook the spatial characteristics of image degradation (regions with richer textures tend to suffer more severe damage), making it hard to achieve the best trade-off between restoration quality and efficiency. To address these issues, we propose a novel texture-aware image restoration method, TAMambaIR, which simultaneously perceives image textures and achieves a trade-off between performance and efficiency. Specifically, we introduce a novel Texture-Aware State Space Model, which enhances texture awareness and improves efficiency by modulating the transition matrix of the state-space equation and focusing on regions with complex textures. Additionally, we design a {Multi-Directional Perception Block} to improve multi-directional receptive fields while maintaining low computational overhead. Extensive experiments on benchmarks for image super-resolution, deraining, and low-light image enhancement demonstrate that TAMambaIR achieves state-of-the-art performance with significantly improved efficiency, establishing it as a robust and efficient framework for image restoration.

图像修复状态空间模型纹理感知Mamba

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。