arXiv:2501.13353cs.CV2025-01被引 1

融合Transformer与状态空间模型,提升图像超分辨率精度与上下文建模能力。

Contrast: A Hybrid Architecture of Transformers and State Space Models for Low-Level Vision

  • 设计混合架构,结合Transformer的精确建模与Mamba的线性计算优势。
  • 在Real-ESRGAN数据集上达到43.27dB PSNR,优于纯Transformer与Mamba模型。
  • 适合追求高精度图像重建的视觉任务研究者使用。

Transformer在图像超分辨率(SR)任务中因强大的全局上下文建模能力而受到青睐,但其二次方计算复杂度需依赖窗口注意力机制,限制了感受野扩展。近期,Mamba架构凭借线性复杂度成为替代方案,避免窗口机制并保持大感受野,但在需要高像素级精度的SR任务中,其隐状态机制会近似压缩上下文,导致误差。为此,本文提出 extbf{Contrast},一种融合卷积、Transformer与状态空间模块的混合架构,有效结合两者的优点,弥补各自不足。通过整合Transformer与状态空间机制,Contrast在增强全局上下文建模的同时提升了像素级精度,在真实图像超分辨率任务中表现更优。

原文摘要 · Abstract (English)

Transformers have become increasingly popular for image super-resolution (SR) tasks due to their strong global context modeling capabilities. However, their quadratic computational complexity necessitates the use of window-based attention mechanisms, which restricts the receptive field and limits effective context expansion. Recently, the Mamba architecture has emerged as a promising alternative with linear computational complexity, allowing it to avoid window mechanisms and maintain a large receptive field. Nevertheless, Mamba faces challenges in handling long-context dependencies when high pixel-level precision is required, as in SR tasks. This is due to its hidden state mechanism, which can compress and store a substantial amount of context but only in an approximate manner, leading to inaccuracies that transformers do not suffer from. In this paper, we propose \textbf{Contrast}, a hybrid SR model that combines \textbf{Con}volutional, \textbf{Tra}nsformer, and \textbf{St}ate Space components, effectively blending the strengths of transformers and Mamba to address their individual limitations. By integrating transformer and state space mechanisms, \textbf{Contrast} compensates for the shortcomings of each approach, enhancing both global context modeling and pixel-level accuracy. We demonstrate that combining these two architectures allows us to mitigate the problems inherent in each, resulting in improved performance on image super-resolution tasks.

图像超分混合模型状态空间Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。