用点提示实现电子显微图像少标注域适应分割
Prompt-DAS: Annotation-Efficient Prompt Learning for Domain Adaptive Semantic Segmentation of Electron Microscopy Images
- 通过可配置点提示+中心点检测，灵活支持无监督、弱监督与交互式分割
- 在多个挑战性数据集上优于现有无监督与弱监督方法
- 适合标注成本高的生物电镜图像分割任务
从大规模电子显微图像中对众多细胞器实例进行域适应语义分割，是实现少标注学习的有前景方向。受SAM启发，我们提出一种可提示多任务框架Prompt-DAS，可在适配训练和测试阶段灵活使用任意数量的点提示。因此，通过不同提示配置，Prompt-DAS可实现无监督域适应（UDA）、弱监督域适应（WDA）以及测试时的交互式分割。不同于需为每个对象实例提供提示的基础模型SAM，Prompt-DAS仅在小规模数据集上训练，即可利用全部实例的完整点、部分实例的稀疏点，甚至无需点提示，这得益于引入的辅助中心点检测任务。此外，提出一种新型提示引导对比学习，以增强特征判别性。在多个挑战性基准上的全面实验表明，该方法在现有UDA、WDA及基于SAM的方法上均表现更优。
原文摘要 · Abstract (English)
Domain adaptive segmentation (DAS) of numerous organelle instances from large-scale electron microscopy (EM) is a promising way to enable annotation-efficient learning. Inspired by SAM, we propose a promptable multitask framework, namely Prompt-DAS, which is flexible enough to utilize any number of point prompts during the adaptation training stage and testing stage. Thus, with varying prompt configurations, Prompt-DAS can perform unsupervised domain adaptation (UDA) and weakly supervised domain adaptation (WDA), as well as interactive segmentation during testing. Unlike the foundation model SAM, which necessitates a prompt for each individual object instance, Prompt-DAS is only trained on a small dataset and can utilize full points on all instances, sparse points on partial instances, or even no points at all, facilitated by the incorporation of an auxiliary center-point detection task. Moreover, a novel prompt-guided contrastive learning is proposed to enhance discriminative feature learning. Comprehensive experiments conducted on challenging benchmarks demonstrate the effectiveness of the proposed approach over existing UDA, WDA, and SAM-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。