用SAM自动找框提示,提升无源域适应分割效果
Leveraging Segment Anything Model for Source-Free Domain Adaptation via Dual Feature Guided Auto-Prompting
- 基于双特征引导自动搜索精准框提示
- 在3D和2D数据集上优于传统方法
- 适合做分割域适应的工程师和研究者
无源域适应(SFDA)分割旨在仅使用源模型和未标注目标数据,将源域训练的模型适配到目标域。受最近成功的通用分割模型SAM启发,该研究首次探索利用SAM通过自动生成精确边界框提示来实现SFDA。发现现有方法生成的边界框因域差异存在缺陷。为此提出新型双特征引导(DFG)自动提示方法:第一阶段通过特征聚合初步适配源模型并准备特征分布;第二阶段根据两类特征分布特性,分别以目标模型特征和SAM特征为引导,逐步扩展框提示以应对类别簇状和分散的目标特征。为消除目标模型过自信预测导致的误检区域,进一步对SAM生成的伪标签进行连通性分析后处理。在3D和2D数据集上的实验表明,该方法性能显著优于传统方法。代码已开源。
原文摘要 · Abstract (English)
Source-free domain adaptation (SFDA) for segmentation aims at adapting a model trained in the source domain to perform well in the target domain with only the source model and unlabeled target data. Inspired by the recent success of Segment Anything Model (SAM) which exhibits the generality of segmenting images of various modalities and in different domains given human-annotated prompts like bounding boxes or points, we for the first time explore the potentials of Segment Anything Model for SFDA via automatedly finding an accurate bounding box prompt. We find that the bounding boxes directly generated with existing SFDA approaches are defective due to the domain gap. To tackle this issue, we propose a novel Dual Feature Guided (DFG) auto-prompting approach to search for the box prompt. Specifically, the source model is first trained in a feature aggregation phase, which not only preliminarily adapts the source model to the target domain but also builds a feature distribution well-prepared for box prompt search. In the second phase, based on two feature distribution observations, we gradually expand the box prompt with the guidance of the target model feature and the SAM feature to handle the class-wise clustered target features and the class-wise dispersed target features, respectively. To remove the potentially enlarged false positive regions caused by the over-confident prediction of the target model, the refined pseudo-labels produced by SAM are further postprocessed based on connectivity analysis. Experiments on 3D and 2D datasets indicate that our approach yields superior performance compared to conventional methods. Code is available at https://github.com/xmed-lab/DFG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。