用不确定性加权特征,让SAM在少标注下高效选图分割
SUGFW+: An Uncertainty-guided Feature Weighting Framework for Cold Start Active Adaptation of SAM in Medical Image Segmentation

- 基于SAM提取图像块级特征与不确定度,生成有区分性的整体特征
- 结合多样性与不确定度,用贪心策略选出最值得标注的样本
- 边选边调优,使模型训练与样本选择紧密协同,适合医疗图像场景
冷启动主动学习(CSAL)能在低标注预算下提升医学图像分割模型性能,通过从大量未标注数据中选取少量样本进行标注。现有方法多依赖效率低下的特定数据集自监督学习来构建特征空间进行样本选择。近年来,基础模型如分割一切模型(SAM)提供了新路径——其预训练特征嵌入具有强泛化能力,微调后可实现高性能下游任务。然而,如何系统利用SAM的内在特征,在低标注预算下完成冷启动样本选择与适配仍不明确。为此,本文提出SUGFW+框架:首先利用SAM进行图像块级特征与不确定度计算(PFUC),再引入基于图像块的全局判别表示(PGDR)模块,将块级嵌入聚合为高区分性、含不确定度感知的图像级特征;随后采用结合聚类与不确定度的贪心选择策略(GSCU)实现样本筛选。不同于以往将样本选择与训练解耦的方法,SUGFW+通过不确定性引导的微调(UPFT)过程,将二者紧密耦合。在四个公开数据集上的实验表明,SUGFW+在现有CSAL方法中表现最优。代码已开源。
原文摘要 · Abstract (English)
Cold Start Active Learning (CSAL) is important in improving the performance of a medical image segmentation model with low annotation budget by querying a small subset for annotation from an unlabeled training set. Existing CSAL methods typically rely on inefficient dataset-specific Self-Supervised Learning (SSL) to map the unlabeled images into a feature space for sample selection. Recently, the advent of foundation models such as the Segment Anything Model (SAM) offer a promising alternative as the pre-trained model can provide strong generalizable feature embeddings, and allow high performance in downstream tasks after fine-tuning (adaptation). However, how to systematically exploit SAM's inherent embeddings for cold-start sample selection during adaptation with low annotation budget remains underexplored. To address this, we propose an extended SAM-based Uncertainty-guided Feature Weighting (SUGFW+) framework for CSAL and adaptation of SAM. Specifically, it leverages the SAM for Patch-level Feature and Uncertainty Calculation (PFUC), and introduces a Patch-based Global Distinct Representation (PGDR) module that aggregates patch-level embeddings into highly discriminative, uncertainty-aware image-level features. These features are then utilized by a Greedy Selection with Cluster and Uncertainty (GSCU) strategy to combine diversity and uncertainty during sample selection. Unlike prior CSAL methods that decouple sample selection from model training, SUGFW+ tightly integrates these two stages via an Uncertainty-Prompted Fine-Tuning (UPFT) process of SAM in model training. Extensive experiments on four public datasets demonstrate that SUGFW+ achieves state-of-the-art performance against existing CSAL methods. Code is available at https://github.com/HiLab-git/SUGFW-plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。