提出可自适应扫描与记忆的视觉模型,提升图像修复质量与效率。
VAMamba: An Efficient Visual Adaptive Mamba for Image Restoration
- 用动态缓存和相似性融合增强特征学习
- 根据像素重要性自适应调整扫描路径,减少无效计算
- 适合需要高效高精度修复的图像处理场景
基于Mamba的图像修复方法虽表现良好,但受限于固定扫描模式和低效特征利用。传统Mamba架构依赖预设路径,无法适应多样退化,制约修复效果与计算效率。为此,我们提出VAMamba,包含两项核心创新:首先,QCLAM(基于队列的缓存低秩自适应记忆)通过先进先出缓存存储历史表示,利用当前LoRA适配特征与缓存特征的相似性实现智能融合,支持动态复用并有效控制内存增长;其次,GPS-SS2D(贪心路径扫描SS2D)引入自适应扫描机制,由视觉变压器生成像素重要性得分图,并采用贪心策略确定最优正向与反向扫描路径,替代固定模式,实现目标化特征提取。QCLAM与GPS-SS2D的结合使VAMamba能自适应聚焦退化区域,同时保持高计算效率。在多种修复任务上的大量实验表明,VAMamba在修复质量和效率上均持续优于现有方法,建立了自适应图像修复的新基准。代码已开源:https://github.com/WaterHQH/VAMamba。
原文摘要 · Abstract (English)
Recent Mamba-based image restoration methods have achieved promising results but remain limited by fixed scanning patterns and inefficient feature utilization. Conventional Mamba architectures rely on predetermined paths that cannot adapt to diverse degradations, constraining both restoration performance and computational efficiency. To overcome these limitations, we propose VAMamba, a Visual Adaptive Mamba framework with two key innovations. First, QCLAM(Queue-basedCacheLow-rankAdaptiveMemory)enhancesfeaturelearningthrougha FIFO cache that stores historical representations. Similarity between current LoRA-adapted and cached features guides intelligent fusion, enabling dynamic reuse while effectively controlling memorygrowth.Second, GPS-SS2D(GreedyPathScanSS2D)introducesadaptive scanning. A Vision Transformer generates score maps to estimate pixel importance, and a greedy strategy de termines optimal forward and backward scanning paths. These learned trajectories replace rigid patterns, enabling SS2D to perform targeted feature extraction. The integration of QCLAM and GPS-SS2D allows VAMamba to adaptively focus on degraded regions while maintaining high computational efficiency. Extensive experiments across diverse restoration tasks demonstrate that VAMamba consistently outperforms existing approaches in both restoration quality and efficiency, establishing new benchmarks for adaptive image restoration. Our code is available at https://github.com/WaterHQH/VAMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。