用CLIP模型实现少样本医学异常检测，图像和像素级效果都更好。
MadCLIP: Few-shot Medical Anomaly Detection with CLIP
- 双分支结构分别学习正常与异常特征，用可学习提示对齐图文语义。
- 在多个数据集上优于现有方法，跨数据集检测准确率提升12.3%。
- 无需合成数据或记忆库，适合医疗场景下标注稀缺的异常检测。
本文提出一种基于预训练CLIP模型的少样本医学异常检测新方法，适用于图像级异常分类（AC）和像素级异常分割（AS）。采用双分支结构，在CLIP视觉编码器中引入可学习适配器，分别捕捉正常与异常特征；通过可学习文本提示增强视觉特征与语义描述之间的对齐。为解决图像与未配对文本提示间的多对一关系，首次在医学领域应用SigLIP损失函数。该方法在多种医学影像模态上验证，无论同数据集还是跨数据集评估均表现优异，显著超越现有方法。相比以往工作，不依赖合成数据或记忆库，消融实验证实各组件有效性。代码已开源：https://github.com/mahshid1998/MadCLIP。
原文摘要 · Abstract (English)
An innovative few-shot anomaly detection approach is presented, leveraging the pre-trained CLIP model for medical data, and adapting it for both image-level anomaly classification (AC) and pixel-level anomaly segmentation (AS). A dual-branch design is proposed to separately capture normal and abnormal features through learnable adapters in the CLIP vision encoder. To improve semantic alignment, learnable text prompts are employed to link visual features. Furthermore, SigLIP loss is applied to effectively handle the many-to-one relationship between images and unpaired text prompts, showcasing its adaptation in the medical field for the first time. Our approach is validated on multiple modalities, demonstrating superior performance over existing methods for AC and AS, in both same-dataset and cross-dataset evaluations. Unlike prior work, it does not rely on synthetic data or memory banks, and an ablation study confirms the contribution of each component. The code is available at https://github.com/mahshid1998/MadCLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。