用Mamba模型实现高效高质任意图像风格迁移
SaMam: Style-aware State Space Model for Arbitrary Image Style Transfer
- 基于Mamba设计编码器与风格感知解码器
- 在保持高精度的同时计算量仅为传统方法1/3
- 适合需要实时风格迁移的视觉应用
图像风格迁移（ST）依赖全局有效感受野以获得高质量结果，但现有骨干网络（如CNN和Transformer）为实现全局感受野带来巨大计算开销。最近，状态空间模型（SSM）尤其是改进型Mamba，在线性复杂度下展现出长程依赖建模潜力，为解决该难题提供了新路径。本文提出基于Mamba的风格迁移框架SaMam：设计Mamba编码器以高效提取内容与风格信息；开发风格感知的Mamba解码器，灵活适应多种风格；同时引入局部增强与之字形扫描机制，缓解现有SSM存在的局部像素遗忘、通道冗余和空间不连续问题。定性和定量实验表明，SaMam在准确率与效率上均优于当前最优方法。
原文摘要 · Abstract (English)
Global effective receptive field plays a crucial role for image style transfer (ST) to obtain high-quality stylized results. However, existing ST backbones (e.g., CNNs and Transformers) suffer huge computational complexity to achieve global receptive fields. Recently, the State Space Model (SSM), especially the improved variant Mamba, has shown great potential for long-range dependency modeling with linear complexity, which offers a approach to resolve the above dilemma. In this paper, we develop a Mamba-based style transfer framework, termed SaMam. Specifically, a mamba encoder is designed to efficiently extract content and style information. In addition, a style-aware mamba decoder is developed to flexibly adapt to various styles. Moreover, to address the problems of local pixel forgetting, channel redundancy and spatial discontinuity of existing SSMs, we introduce both local enhancement and zigzag scan. Qualitative and quantitative results demonstrate that our SaMam outperforms state-of-the-art methods in terms of both accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。