arXiv:2409.00556cs.CV2024-09中稿 · BMVC 2024被引 14

用大模型实现零样本工业缺陷检测,精度超现有方法。

FADE: Few-shot/zero-shot Anomaly Detection Engine using Large Vision-Language Model

  • 基于CLIP改进多尺度图像嵌入,增强图文对齐能力。
  • 零样本下像素AUROC达89.6%(MVTec-AD),1样本时达95.4%。
  • 适合缺乏正常样本的工业质检场景,无需重新训练。

自动图像异常检测在制造业质量检验中至关重要。传统无监督方法需为每类物体收集大量正常样本进行训练,但在真实场景中往往只有零个或少量正常样本可用,导致建模困难。近期大型视觉语言模型在下游任务中展现出强大的零样本性能,但未专门针对异常检测优化。本文提出少样本/零样本异常检测引擎FADE,利用CLIP模型并针对性改进:1)调整CLIP以提取更契合语言语义的多尺度图像块嵌入;2)自动生成与工业异常检测相关的文本提示集成;3)引入查询与参考图像的视觉引导,进一步提升零样本和少样本检测效果。在MVTec-AD(及VisA)数据集上,FADE在零样本下的像素AUROC达到89.6%(91.5%),1正常样本时提升至95.4%(97.5%)。代码已开源。

原文摘要 · Abstract (English)

Automatic image anomaly detection is important for quality inspection in the manufacturing industry. The usual unsupervised anomaly detection approach is to train a model for each object class using a dataset of normal samples. However, a more realistic problem is zero-/few-shot anomaly detection where zero or only a few normal samples are available. This makes the training of object-specific models challenging. Recently, large foundation vision-language models have shown strong zero-shot performance in various downstream tasks. While these models have learned complex relationships between vision and language, they are not specifically designed for the tasks of anomaly detection. In this paper, we propose the Few-shot/zero-shot Anomaly Detection Engine (FADE) which leverages the vision-language CLIP model and adjusts it for the purpose of industrial anomaly detection. Specifically, we improve language-guided anomaly segmentation 1) by adapting CLIP to extract multi-scale image patch embeddings that are better aligned with language and 2) by automatically generating an ensemble of text prompts related to industrial anomaly detection. 3) We use additional vision-based guidance from the query and reference images to further improve both zero-shot and few-shot anomaly detection. On the MVTec-AD (and VisA) dataset, FADE outperforms other state-of-the-art methods in anomaly segmentation with pixel-AUROC of 89.6% (91.5%) in zero-shot and 95.4% (97.5%) in 1-normal-shot. Code is available at https://github.com/BMVC-FADE/BMVC-FADE.

异常检测视觉语言模型少样本学习工业质检

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。