用知识驱动提示学习,实现无需训练样本的高精度异常检测
KAnoCLIP: Zero-Shot Anomaly Detection through Knowledge-Driven Prompt Learning and Enhanced Cross-Modal Integration
- 用大语言模型和视觉问答系统生成可学习的异常提示,替代固定文本
- 在12个工业与医疗数据集上达到当前最佳效果,像素级检测更精准
- 适合缺乏标注数据的场景,如医疗影像或隐私敏感领域
零样本异常检测(ZSAD)无需目标数据集的训练样本,适用于隐私敏感或数据有限的场景。尽管视觉语言模型如CLIP在ZSAD中展现潜力,但仍存在局限:依赖人工设计的固定文本描述或异常提示,耗时且易产生语义模糊;同时,CLIP在像素级异常分割上表现不佳,更关注全局语义而非局部细节。为此,我们提出KAnoCLIP框架,结合大型语言模型(GPT-3.5)的通用知识与视觉问答系统(Llama3)的图像特定细粒度知识,通过知识驱动提示学习(KnPL)生成可学习的异常提示,消除对固定文本提示的依赖,提升泛化能力。KAnoCLIP包含基于视觉-视觉注意力(CLIP-VV)的CLIP视觉编码器、双向多层级跨模态交互(Bi-CMCI)以及卷积适配器(Conv-Adapter),有效保留局部视觉语义,增强局部跨模态融合,并对齐全局视觉特征与文本信息,显著提升像素级异常检测性能。KAnoCLIP在12个工业与医学数据集上均达到最新最优表现,证明其卓越泛化能力。
原文摘要 · Abstract (English)
Zero-shot anomaly detection (ZSAD) identifies anomalies without needing training samples from the target dataset, essential for scenarios with privacy concerns or limited data. Vision-language models like CLIP show potential in ZSAD but have limitations: relying on manually crafted fixed textual descriptions or anomaly prompts is time-consuming and prone to semantic ambiguity, and CLIP struggles with pixel-level anomaly segmentation, focusing more on global semantics than local details. To address these limitations, We introduce KAnoCLIP, a novel ZSAD framework that leverages vision-language models. KAnoCLIP combines general knowledge from a Large Language Model (GPT-3.5) and fine-grained, image-specific knowledge from a Visual Question Answering system (Llama3) via Knowledge-Driven Prompt Learning (KnPL). KnPL uses a knowledge-driven (KD) loss function to create learnable anomaly prompts, removing the need for fixed text prompts and enhancing generalization. KAnoCLIP includes the CLIP visual encoder with V-V attention (CLIP-VV), Bi-Directional Cross-Attention for Multi-Level Cross-Modal Interaction (Bi-CMCI), and Conv-Adapter. These components preserve local visual semantics, improve local cross-modal fusion, and align global visual features with textual information, enhancing pixel-level anomaly detection. KAnoCLIP achieves state-of-the-art performance in ZSAD across 12 industrial and medical datasets, demonstrating superior generalization compared to existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。