用注意力重加权实现跨架构的高效模型适配，参数量不到5%。
MSLoRA: Multi-Scale Low-Rank Adaptation via Attention Reweighting
- 通过多尺度非线性变换重加权特征响应，不修改主干权重
- 在分类、检测、分割任务上提升性能，仅用主干参数的4.8%左右
- 适用于CNN和ViT，优化稳定且收敛快，适合冻结主干场景
我们提出MSLoRA，一种与主干网络无关的参数高效适配器，通过重加权特征响应而非微调底层主干。现有低秩适配方法多局限于视觉变压器（ViTs），难以跨架构泛化。MSLoRA通过低秩线性投影与多尺度非线性变换的结合，联合调节空间与通道注意力，二者通过逐点乘法与残差连接融合，形成轻量模块，在保持预训练权重冻结的同时调整特征注意力。大量实验表明，MSLoRA在分类、检测和分割任务上均持续提升迁移性能，参数量约为主干的5%以下。该设计还实现了稳定的优化、快速收敛及强跨架构泛化能力。通过重加权而非重调，MSLoRA为冻结视觉主干提供了简单通用的高效适配方案。
原文摘要 · Abstract (English)
We introduce MSLoRA, a backbone-agnostic, parameter-efficient adapter that reweights feature responses rather than re-tuning the underlying backbone. Existing low-rank adaptation methods are mostly confined to vision transformers (ViTs) and struggle to generalize across architectures. MSLoRA unifies adaptation for both convolutional neural networks (CNNs) and ViTs by combining a low-rank linear projection with a multi-scale nonlinear transformation that jointly modulates spatial and channel attention. The two components are fused through pointwise multiplication and a residual connection, yielding a lightweight module that shifts feature attention while keeping pretrained weights frozen. Extensive experiments demonstrate that MSLoRA consistently improves transfer performance on classification, detection, and segmentation tasks with roughly less than 5\% of backbone parameters. The design further enables stable optimization, fast convergence, and strong cross-architecture generalization. By reweighting rather than re-tuning, MSLoRA provides a simple and universal approach for efficient adaptation of frozen vision backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。