用Mamba模型提升视频超分辨率,处理长序列更高效。
VSRM: A Robust Mamba-Based Framework for Video Super-Resolution
- 引入时空双向Mamba模块,捕捉长时序依赖
- 在多个数据集上达到最优性能,峰值信噪比领先
- 适合需要高帧率视频增强的研究与应用
视频超分辨率是低层视觉中的重要挑战。现有基于CNN和Transformer的方法虽取得显著成果,但前者受限于局部感受野,后者面临二次复杂度问题,难以处理长序列。近期,Mamba因其线性复杂度和大感受野受到关注。本文提出VSRM,一种基于Mamba的新型视频超分辨率框架。VSRM引入空间到时间与时间到空间的Mamba模块,高效提取长程时空特征并扩展感受野。为更好对齐相邻帧,设计可变形跨Mamba对齐模块,使补偿过程更动态灵活,避免特征畸变。此外,提出简单有效的频率域Charbonnier-like损失,缩小重建帧与真实帧的频域差异,更好保留高频内容,提升视觉质量。大量实验表明,VSRM在多个基准测试中达到当前最优性能,为后续研究奠定坚实基础。
原文摘要 · Abstract (English)
Video super-resolution remains a major challenge in low-level vision tasks. To date, CNN- and Transformer-based methods have delivered impressive results. However, CNNs are limited by local receptive fields, while Transformers struggle with quadratic complexity, posing challenges for processing long sequences in VSR. Recently, Mamba has drawn attention for its long-sequence modeling, linear complexity, and large receptive fields. In this work, we propose VSRM, a novel \textbf{V}ideo \textbf{S}uper-\textbf{R}esolution framework that leverages the power of \textbf{M}amba. VSRM introduces Spatial-to-Temporal Mamba and Temporal-to-Spatial Mamba blocks to extract long-range spatio-temporal features and enhance receptive fields efficiently. To better align adjacent frames, we propose Deformable Cross-Mamba Alignment module. This module utilizes a deformable cross-mamba mechanism to make the compensation stage more dynamic and flexible, preventing feature distortions. Finally, we minimize the frequency domain gaps between reconstructed and ground-truth frames by proposing a simple yet effective Frequency Charbonnier-like loss that better preserves high-frequency content and enhances visual quality. Through extensive experiments, VSRM achieves state-of-the-art results on diverse benchmarks, establishing itself as a solid foundation for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。