让分割模型自动找医学图像中的关键概念,无需重训练或人工写提示。
Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation

- 用可学习的嵌入向量在冻结模型内搜索可迁移的视觉概念。
- 仅需少量标注样本,即实现当前最优的少样本分割性能。
- 自动标记不确定区域供人工修正,形成持续优化的标注闭环。
将基础分割模型从依赖人工提示的工具转变为全自动标注器,对医疗数据规模化标注至关重要。现有方法通常依赖外部特征匹配器或辅助网络来自动化几何提示,但引入额外结构开销并限制性能扩展性。尽管SAM3原生支持通过可复用文本提示进行概念分割,其在医学影像中的直接应用受限于缺乏细粒度临床知识以及人类描述的模糊性。本文提出Mask to Concept(M2C),一种无需外部模块、参数重训练或手动文本工程的高效框架,适配SAM3用于医学少样本标注。仅需少量标注图像,M2C即可在冻结架构内自动搜索可迁移的视觉概念:初始化可学习概念嵌入,用其提示分割,并通过最小化概念分割误差的梯度更新嵌入。我们进一步引入混合不确定性估计(HUE)模块,计算预测熵并将概念预测映射回框提示,衡量概念-几何提示不一致性。高不确定性样本被主动标记以供人工修正,修正后的掩码再反馈至M2C,持续搜索更精确的概念嵌入,形成低人力投入的自增强标注循环。在多个医学分割基准上实验表明,该方法达到当前最优的少样本分割性能与卓越的标注效率,为可扩展的医学图像标注提供实用高效路径。代码见:https://github.com/Huster-Hq/M2C。
原文摘要 · Abstract (English)
Transforming foundation segmentation models from human-prompted tools into auto-promptable annotators is critical for scalable medical data annotation. Current methods commonly depend on external feature matchers or auxiliary networks to automate geometric prompting, but introducing architectural overhead and limiting performance scalability. Although SAM3 natively supports concept segmentation via reusable text prompts, its direct use in medical imaging is hindered by a lack of fine-grained clinical knowledge and the ambiguity of human-written descriptions. In this work, we propose Mask to Concept (M2C), an efficient framework that adapts SAM3 for medical few-shot annotation without external modules, parameter retraining, or manual text engineering. Using only a few labeled images, M2C enables SAM3 to automatically search for transferable visual concepts entirely within its frozen architecture: it initializes a learnable concept embedding, uses it to prompt segmentation, and updates the embedding by gradients of minimizing the concept segmentation error. We further introduce a Hybrid Uncertainty Estimation (HUE) module that calculates the prediction entropy and maps concept predictions back to the box prompts, measuring concept-geometry prompting inconsistency. Highly uncertain samples are flagged actively for human correction, and the corrected masks are then fed back to M2C to continuously search for more precise concept embeddings, forming a self-enhancing annotation loop with minimal expert effort. Experiments on medical segmentation benchmarks show that our method achieves SOTA few-shot segmentation performance and outstanding annotation efficiency, offering a practical and efficient pathway toward scalable medical image labeling. Codes are at https://github.com/Huster-Hq/M2C.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。