arXiv:2606.01947cs.CVcs.AI2026-06被引 13

用少量参数微调大模型,实现高效实例分割。

Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks

论文配图:Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
图 1 · 摘自论文原文
  • 在Transformer模型中引入适配器和低秩微调(LoRA)。
  • 仅微调1-6%参数,性能媲美传统方法(需40-55%)。
  • 适合资源有限但需高效迁移学习的实例分割场景。

人工智能研究与应用正因大规模预训练模型的发展而转向,这些模型在众多任务上达到顶尖水平。然而,参数量激增带来了对参数高效训练策略的需求。尽管已有显著进展,针对基于Transformer的模型在实例分割任务中的参数高效微调(PEFT)研究仍有限。本研究系统评估了适配器和低秩微调(LoRA)在两种模型、四个基准数据集上的有效性。将顺序排列的适配器模块与首次应用于可变形注意力的LoRA结合,仅需微调约1-6%的参数即可取得竞争性性能,显著优于传统微调所需的40-55%。关键发现表明,每个Transformer块使用2-3个适配器能实现性能与效率的最佳平衡。此外,将LoRA应用于可变形注意力时表现出强参数效率,某些情况下甚至优于适配器配置。结果表明,PEFT的效果随数据集复杂度与模型架构变化,强调了上下文特定调优的重要性。总体而言,本工作展示了PEFT在实例分割任务中实现可扩展、可定制、计算高效的迁移学习的巨大潜力。

原文摘要 · Abstract (English)

Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of transformer-based models for instance segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable attention--explored here for the first time--achieves competitive performance while fine-tuning only about 1-6% of model parameters, a marked improvement over the 40-55% required in traditional fine-tuning. Key findings indicate that using 2-3 adapters per transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance segmentation tasks.

实例分割参数高效LoRATransformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。