通过聚焦异常区域增强视觉表征,实现无需训练的精准缺陷检测。
AF-CLIP: Zero-Shot Anomaly Detection via Anomaly-Focused CLIP Adaptation
- 引入轻量适配器强化异常相关特征,同时优化图像级与局部块级表示。
- 在多个工业与医疗数据集上达到优于现有方法的零样本检测准确率。
- 适合缺乏标注数据的工业质检和医学影像异常发现场景。
视觉异常检测广泛应用于工业质检与医学诊断。现有方法通常需要大量训练样本,难以适应零样本或少样本场景。尽管近期研究利用CLIP的零样本识别能力解决此问题,但往往忽视对视觉特征的优化以聚焦局部异常,影响检测效果。本文提出AF-CLIP(异常聚焦的CLIP),通过显著增强视觉表征来关注局部缺陷。方法引入轻量级适配器,强调视觉特征中与异常相关的模式,同时优化类别级特征用于图像分类、块级特征用于精确定位。为捕捉不同尺寸的异常并提升检测精度,在适配器前设计多尺度空间聚合机制,有效整合邻域上下文信息。此外,我们设计可学习的文本提示,通用描述正常与异常状态。在辅助数据集上通过复合目标函数优化后,AF-CLIP展现出强大的零样本检测能力。该方法还可通过额外记忆库扩展至少样本场景。在多种工业与医疗数据集上的实验结果证明了其有效性与泛化能力。代码已开源:https://github.com/Faustinaqq/AF-CLIP。
原文摘要 · Abstract (English)
Visual anomaly detection has been widely used in industrial inspection and medical diagnosis. Existing methods typically demand substantial training samples, limiting their utility in zero-/few-shot scenarios. While recent efforts have leveraged CLIP's zero-shot recognition capability for this task, they often ignore optimizing visual features to focus on local anomalies, reducing their efficacy. In this work, we propose AF-CLIP (Anomaly-Focused CLIP) by dramatically enhancing its visual representations to focus on local defects. Our approach introduces a lightweight adapter that emphasizes anomaly-relevant patterns in visual features, simultaneously optimizing both class-level features for image classification and patch-level features for precise localization. To capture anomalies of different sizes and improve detection accuracy, prior to the adapter, we develop a multi-scale spatial aggregation mechanism to effectively consolidate neighborhood context. Complementing these visual enhancements, we design learnable textual prompts that generically characterize normal and abnormal states. After optimization on auxiliary datasets using a composite objective function, AF-CLIP demonstrates strong zero-shot detection capability. Our method is also extended to few-shot scenarios by extra memory banks. Experimental results across diverse industrial and medical datasets demonstrate the effectiveness and generalization of our proposed method. Code is available at https://github.com/Faustinaqq/AF-CLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。