用跨层协同的Mamba适配器，高效微调图像压缩模型用于视觉任务。
CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression
- 引入Mamba适配器与多尺度分支，捕捉局部特征和全局依赖。
- 设计共享参数的跨层融合模块，减少冗余，参数量降低72%。
- 适合需轻量化微调预训练压缩模型的视觉任务研究者。
为降低部署成本和重训练开销，将预训练图像压缩模型适配到下游机器视觉任务受到广泛关注。然而，现有方法通常独立插入微调模块至冻结主干网络，缺乏显式的跨层协调机制。为此，我们提出一种新框架CrossMambaTuning，结合状态空间模型与跨层交互机制，实现参数高效的微调。具体而言，设计了一个配备任务特定提示和多尺度分支的高效Mamba适配器，以精确捕捉局部特征与全局依赖；同时引入尺度不变的跨层适配器（SICA），采用参数共享策略在不同尺度间融合任务信息，减少冗余。大量实验表明，CrossMambaTuning在多个机器视觉任务上达到当前最优性能，相比最先进方法参数开销降低72%。代码已开源：https://github.com/rsr1123/CrossMambaTuning。
原文摘要 · Abstract (English)
To reduce deployment cost and retraining overhead, adapting pretrained learned image compression (LIC) models to downstream machine vision tasks has attracted growing attention. However, existing methods typically insert fine-tuning modules independently into frozen backbones, lacking explicit mechanisms for cross-layer coordination. To address this limitation, we propose a novel framework named CrossMambaTuning, which integrates State Space Models with cross-layer interaction mechanisms for parameter-efficient fine-tuning. Specifically, we design an efficient Mamba adapter equipped with task-specific prompts and multi-scale branching to precisely capture both local features and global dependencies. Furthermore, we introduce a Scale-Invariant Cross-Layer Adapter (SICA) utilizing a parameter-sharing strategy to fuse task information across different scales and reduce redundancy. Extensive experiments demonstrate that CrossMambaTuning achieves state-of-the-art (SOTA) performance on multiple machine vision tasks, reducing parameter overhead by 72\% compared to SOTA methods. Code is available at https://github.com/rsr1123/CrossMambaTuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。