通过分离全局与局部提示,实现无需训练样本的异常检测。
GlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection
- 分离全局与局部文本提示,避免依赖特定物体。
- 在15个真实数据集上优于现有方法,工业与医疗领域均表现优异。
- 适合无标注异常检测场景,尤其数据稀缺或分布差异大的情况。
零样本异常检测(ZSAD)在目标数据集无训练样本时至关重要,尤其适用于目标域与训练数据分布不同或数据受限的场景。尽管预训练视觉语言模型在各类视觉任务中表现强劲,但其侧重于学习类别语义,直接用于ZSAD存在挑战。为此,我们提出GlocalCLIP,首次将全局与局部提示分离,并联合优化。该方法使对象无关的全局-局部语义提示能有效捕捉通用正常与异常模式,不依赖图像中特定物体。通过在文本编码器中采用深度文本提示调优,精细化调整文本提示;在视觉编码器中引入V-V注意力层,捕获细粒度局部图像特征。最后,设计全局-局部对比学习机制,增强两者互补性,提升跨领域异常检测能力。GlocalCLIP在15个来自工业与医疗领域的现实数据集上验证了卓越的泛化性能,显著优于现有方法。代码将公开于https://github.com/YUL-git/GlocalCLIP。
原文摘要 · Abstract (English)
Zero-shot anomaly detection (ZSAD) is crucial for detecting anomalous patterns in target datasets without using training samples, specifically in scenarios where there are distributional differences between the target domain and training data or where data scarcity arises because of restricted access. Although recently pretrained vision-language models demonstrate strong zero-shot performance across various visual tasks, they focus on learning class semantics, which makes their direct application to ZSAD challenging. To address this scenario, we propose GlocalCLIP, which uniquely separates global and local prompts and jointly optimizes them. This approach enables the object-agnostic glocal semantic prompt to effectively capture general normal and anomalous patterns without dependency on specific objects in the image. We refine the text prompts for more precise adjustments by utilizing deep-text prompt tuning in the text encoder. In the vision encoder, we apply V-V attention layers to capture detailed local image features. Finally, we introduce glocal contrastive learning to improve the complementary learning of global and local prompts, effectively detecting anomalous patterns across various domains. The generalization performance of GlocalCLIP in ZSAD was demonstrated on 15 real-world datasets from both the industrial and medical domains, achieving superior performance compared to existing methods. Code will be made available at https://github.com/YUL-git/GlocalCLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。