用视觉语言模型实现零样本缺陷检测,无需标注数据
Analyzing VLM-Based Approaches for Anomaly Classification and Segmentation
- 通过图像与文本对齐,用自然语言描述正常/异常状态
- 在MVTec AD和VisA数据集上达到高精度分类与分割性能
- 适合工业质检场景,为方法选型提供实证指导
视觉语言模型(VLM),特别是CLIP,通过学习图像与文本的对齐表示,实现了无需大量标注数据的零样本和少样本缺陷识别,革新了异常检测。本文系统分析了基于VLM的异常分类(AC)与异常分割(AS)方法,涵盖滑动窗口密集特征提取(WinCLIP)、多阶段特征对齐与可学习投影(AprilLab框架)以及组合提示集成策略。在MVTec AD和VisA等基准上,评估了特征提取机制、文本-视觉对齐策略、提示工程、零样本与少样本权衡、计算效率及跨域泛化能力。实验对比了分类准确率、分割精度与推理效率。主要贡献在于揭示VLM在异常检测中成功的关键机制,为工业质量控制中的方法选择提供实践洞察,并指明当前局限与未来研究方向。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs), particularly CLIP, have revolutionized anomaly detection by enabling zero-shot and few-shot defect identification without extensive labeled datasets. By learning aligned representations of images and text, VLMs facilitate anomaly classification and segmentation through natural language descriptions of normal and abnormal states, eliminating traditional requirements for task-specific training or defect examples. This project presents a comprehensive analysis of VLM-based approaches for anomaly classification (AC) and anomaly segmentation (AS). We systematically investigate key architectural paradigms including sliding window-based dense feature extraction (WinCLIP), multi-stage feature alignment with learnable projections (AprilLab framework), and compositional prompt ensemble strategies. Our analysis evaluates these methods across critical dimensions: feature extraction mechanisms, text-visual alignment strategies, prompt engineering techniques, zero-shot versus few-shot trade-offs, computational efficiency, and cross-domain generalization. Through rigorous experimentation on benchmarks such as MVTec AD and VisA, we compare classification accuracy, segmentation precision, and inference efficiency. The primary contribution is a foundational understanding of how and why VLMs succeed in anomaly detection, synthesizing practical insights for method selection and identifying current limitations. This work aims to facilitate informed adoption of VLM-based methods in industrial quality control and guide future research directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。