用视觉语言模型统一工业缺陷检测、定位与描述,支持零样本多类型识别。
GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models

- 引入双分割标记[SEG_defect]和[SEG_normal],实现语言引导的像素级定位。
- 在VisA和Real-IAD数据集上图像级检测性能优于现有零样本CLIP方法。
- 一次模型完成检测、分割、分类与可解释描述,适合工业质检场景应用。
工业检测不仅需要判断异常存在,还需定位缺陷区域、识别缺陷类型并提供可解释的视觉证据。现有基于CLIP的方法虽能良好检测与定位异常,但缺乏语言层面的缺陷理解;而指令微调的视觉语言模型虽可描述缺陷,却无法原生生成像素级掩码。我们提出GenAU,一种面向工业异常理解的通用视觉语言框架,将图像级检测、像素级分割、多类型异常检测与缺陷分析统一于单一指令跟随模型中。GenAU通过在视觉语言模型中引入两个分割标记[SEG_defect]和[SEG_normal],其隐藏状态作为语言引导的查询,在多尺度视觉特征上进行像素级定位;图像级得分融合该掩码与解码器的文本正常/异常判断,语言解码器则生成结构化缺陷感知响应。通过联合语言建模与分割目标训练,GenAU在一个架构与流程中涵盖四项任务,在检测与分割成本可控的前提下,实现了零样本多类型检测与语言引导的缺陷分析。在跨数据集基准测试中,GenAU在VisA和Real-IAD上达到所有基于CLIP的零样本方法中最强的图像级检测性能,分割表现接近但未超越专用的CLIP基线。
原文摘要 · Abstract (English)
Industrial inspection requires more than binary anomaly detection: a practical system should determine whether an anomaly exists, localize the defective region, identify the defect type, and provide interpretable visual evidence. Existing CLIP-based methods detect and localize anomalies well but offer limited language-level defect understanding, while instruction-tuned vision-language models can describe defects but do not natively produce pixel-level masks. We introduce GenAU, a Generalist vision-language framework for industrial Anomaly Understanding that unifies image-level detection, pixel-level segmentation, multi-type anomaly detection, and defect analysis in a single instruction-following model. GenAU augments a vision-language model with two segmentation tokens, [SEG_defect] and [SEG_normal], whose hidden states act as language-grounded queries over multi-scale visual features for pixel-level localization; the image-level score fuses this map with the decoder's textual normal/defect decision, while the language decoder produces structured defect-aware responses. Trained with a joint language-modeling and segmentation objective, GenAU covers all four tasks within one architecture and recipe, adding zero-shot multi-type detection and language-grounded defect analysis at a quantified cost to detection and segmentation. Across cross-dataset benchmarks, GenAU attains the strongest image-level detection among CLIP-based zero-shot methods on VisA and Real-IAD, with segmentation approaching but not surpassing specialized CLIP baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。