用文字描述病灶特征,实现少样本下多类医学异常检测
SD-MAD: Sign-Driven Few-shot Multi-Anomaly Detection in Medical Images
- 基于大语言模型生成病灶文本描述,引导模型识别多类异常
- 两阶段设计:先对齐病灶特征与类别,再自动筛选关键特征
- 适合医疗影像少样本场景,尤其需区分多种异常的诊断任务
医学异常检测对早期临床干预至关重要,但受限于隐私和数据孤岛导致高质量医学影像数据稀缺。少样本学习通过利用视觉-语言模型中的大规模先验知识缓解此问题。现有方法常将正常与异常视为单类分类,忽略多类异常的区别。为此,本文提出SD-MAD框架,面向需识别多类异常的少样本医学异常检测场景。为捕捉各类异常的详细放射学特征,我们利用大语言模型为每类异常生成多样化文本描述,假设不同异常在各类别中可能共享共同影像征象。具体提出两阶段框架:(i) 通过增强类间差异,对齐放射学征象与异常类别;(ii) 在推理时采用自动特征选择策略,缓解因数据有限导致的欠拟合与不确定样本问题。此外,设计三种评估协议全面量化多异常检测性能。大量实验验证了方法的有效性。
原文摘要 · Abstract (English)
Medical anomaly detection (AD) is crucial for early clinical intervention, yet it faces challenges due to limited access to high-quality medical imaging data, caused by privacy concerns and data silos. Few-shot learning has emerged as a promising approach to alleviate these limitations by leveraging the large-scale prior knowledge embedded in vision-language models (VLMs). Recent advancements in few-shot medical AD have treated normal and abnormal cases as a one-class classification problem, often overlooking the distinction among multiple anomaly categories. Thus, in this paper, we propose a framework tailored for few-shot medical anomaly detection in the scenario where the identification of multiple anomaly categories is required. To capture the detailed radiological signs of medical anomaly categories, our framework incorporates diverse textual descriptions for each category generated by a Large-Language model, under the assumption that different anomalies in medical images may share common radiological signs in each category. Specifically, we introduce SD-MAD, a two-stage Sign-Driven few-shot Multi-Anomaly Detection framework: (i) Radiological signs are aligned with anomaly categories by amplifying inter-anomaly discrepancy; (ii) Aligned signs are selected further to mitigate the effect of the under-fitting and uncertain-sample issue caused by limited medical data, employing an automatic sign selection strategy at inference. Moreover, we propose three protocols to comprehensively quantify the performance of multi-anomaly detection. Extensive experiments illustrate the effectiveness of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。