用可扩展状态空间模型实现任意尺度超分辨率,速度快效果好。
$\text{S}^{3}$Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Model
- 设计可扩展状态空间模型,线性复杂度建模连续图像表示。
- 在合成与真实数据集上均达到顶尖性能,支持任意缩放比例。
- 适合需要高效通用超分辨率的工业级图像处理场景。
任意尺度超分辨率(ASSR)旨在仅用单一模型将低分辨率图像放大至任意尺度,克服传统方法仅支持固定缩放倍数(如×2、×4)的局限。隐式神经表示(INR)的兴起为ASSR提供了新思路,通过建模坐标与像素值之间的连续表示空间,实现对原始连续信号的重建。因此,ASSR的核心目标是基于低分辨率输入构建连续表示空间。然而,现有方法主要依赖卷积神经网络(CNN)和变换器(Transformer),面临计算复杂度高、长程依赖建模不足等问题,限制了实际应用。为此,我们提出新型任意尺度超分辨率方法S³Mamba,以构建可扩展的连续表示空间。具体地,提出可扩展状态空间模型(SSSM),在离散化过程中调制状态转移矩阵与采样矩阵,实现线性计算复杂度下的连续表示建模;同时设计新颖的尺度感知自注意力机制,增强网络对不同尺度下全局重要特征的感知能力。在合成与真实世界基准上的大量实验表明,该方法在任意超分辨率尺度下均达到当前最优性能,并具备优异泛化能力。
原文摘要 · Abstract (English)
Arbitrary scale super-resolution (ASSR) aims to super-resolve low-resolution images to high-resolution images at any scale using a single model, addressing the limitations of traditional super-resolution methods that are restricted to fixed-scale factors (e.g., $\times2$, $\times4$). The advent of Implicit Neural Representations (INR) has brought forth a plethora of novel methodologies for ASSR, which facilitate the reconstruction of original continuous signals by modeling a continuous representation space for coordinates and pixel values, thereby enabling arbitrary-scale super-resolution. Consequently, the primary objective of ASSR is to construct a continuous representation space derived from low-resolution inputs. However, existing methods, primarily based on CNNs and Transformers, face significant challenges such as high computational complexity and inadequate modeling of long-range dependencies, which hinder their effectiveness in real-world applications. To overcome these limitations, we propose a novel arbitrary-scale super-resolution method, called $\text{S}^{3}$Mamba, to construct a scalable continuous representation space. Specifically, we propose a Scalable State Space Model (SSSM) to modulate the state transition matrix and the sampling matrix of step size during the discretization process, achieving scalable and continuous representation modeling with linear computational complexity. Additionally, we propose a novel scale-aware self-attention mechanism to further enhance the network's ability to perceive global important features at different scales, thereby building the $\text{S}^{3}$Mamba to achieve superior arbitrary-scale super-resolution. Extensive experiments on both synthetic and real-world benchmarks demonstrate that our method achieves state-of-the-art performance and superior generalization capabilities at arbitrary super-resolution scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。