arXiv:2603.00433cs.CVcs.AI2026-03

用任务感知提示+选择性微调,低成本高效适配视觉大模型做超声多任务分析

TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis

  • 设计任务感知提示和顶层层选择性微调,仅更新少量参数
  • 在超声多任务数据集上达到第五名,性能接近全量微调
  • 适合资源有限但需快速部署多任务医疗影像模型的场景

在医学图像分析中同时执行分割、分类、检测和回归等多任务时,常面临模型泛化能力不足与共享特征表示优化困难的问题。尽管视觉基础模型(VFMs)具备强大通用表征能力,但在有限医疗数据上进行全量微调易过拟合且计算成本高。现有参数高效微调方法多采用无任务区分的适配策略,忽视了任务特异性机制及不同网络层对微调的敏感性差异。本文提出任务感知提示与选择性层微调(TAP-SLF)框架,通过在输入序列中嵌入任务感知软提示以编码任务先验,并仅对编码器顶部特定层应用LoRA进行微调。该策略仅更新少量模型参数,同时保持预训练主干冻结。结合任务感知提示与选择性高层微调,实现基于共享主干的多任务高效适配。在FMC_UIA 2026挑战赛测试集上取得第五名,且在官方发布训练集(8:2划分)上的评估验证了该方法的有效性。

原文摘要 · Abstract (English)

Executing multiple tasks simultaneously in medical image analysis, including segmentation, classification, detection, and regression, often introduces significant challenges regarding model generalizability and the optimization of shared feature representations. While Vision Foundation Models (VFMs) provide powerful general representations, full fine-tuning on limited medical data is prone to overfitting and incurs high computational costs. Moreover, existing parameter-efficient fine-tuning approaches typically adopt task-agnostic adaptation protocols, overlooking both task-specific mechanisms and the varying sensitivity of model layers during fine-tuning. In this work, we propose Task-Aware Prompting and Selective Layer Fine-Tuning (TAP-SLF), a unified framework for multi-task ultrasound image analysis. TAP-SLF incorporates task-aware soft prompts to encode task-specific priors into the input token sequence and applies LoRA to selected specific top layers of the encoder. This strategy updates only a small fraction of the VFM parameters while keeping the pre-trained backbone frozen. By combining task-aware prompts with selective high-layer fine-tuning, TAP-SLF enables efficient VFM adaptation to diverse medical tasks within a shared backbone. Results on the FMC_UIA 2026 Challenge test set, where TAP-SLF wins fifth place, combined with evaluations on the officially released training dataset using an 8:2 train-test split, demonstrate that task-aware prompting and selective layer tuning are effective strategies for efficient VFM adaptation.

多任务学习参数高效医学影像超声分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。