用少量样本和文本提示，让模型快速学会检测产品缺陷。
Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model
- 用视觉语言模型+上下文学习，仅需一张样图即可完成检测
- 在MVTec AD数据集上达到0.804的马修相关系数和0.950的F1分数
- 适合新产线快速部署，无需重新训练模型
我们提出一种基于视觉语言模型（VLM）的通用视觉检测方法，仅需少量正常或缺陷产品的图像及对应解释性文本作为检测标准。尽管现有VLM在多种任务中表现优异，但未针对视觉检测任务进行训练。为此，我们构建了一个包含多样正常与缺陷产品图像的数据集，并采用统一格式的输出文本对VLM进行微调。对于新产品，该方法利用上下文学习机制，通过一张正常或缺陷样本图像及其对应的解释性文本与视觉提示，实现快速检测。该方法避免了大规模样本收集和模型重训练的需求。实验结果表明，在单样本（one-shot）条件下，该方法在MVTec AD数据集上取得了0.804的马修相关系数（MCC）和0.950的F1分数。代码已开源。
原文摘要 · Abstract (English)
We propose general visual inspection model using Vision-Language Model~(VLM) with few-shot images of non-defective or defective products, along with explanatory texts that serve as inspection criteria. Although existing VLM exhibit high performance across various tasks, they are not trained on specific tasks such as visual inspection. Thus, we construct a dataset consisting of diverse images of non-defective and defective products collected from the web, along with unified formatted output text, and fine-tune VLM. For new products, our method employs In-Context Learning, which allows the model to perform inspections with an example of non-defective or defective image and the corresponding explanatory texts with visual prompts. This approach eliminates the need to collect a large number of training samples and re-train the model for each product. The experimental results show that our method achieves high performance, with MCC of 0.804 and F1-score of 0.950 on MVTec AD in a one-shot manner. Our code is available at~https://github.com/ia-gu/Vision-Language-In-Context-Learning-Driven-Few-Shot-Visual-Inspection-Model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。