用视觉语言模型提升异常检测，无需额外数据和训练。
VL4AD: Vision-Language Models Improve Pixel-wise Anomaly Detection
- 引入视觉语言编码器，利用预训练语义知识增强异常感知。
- 通过文本提示实现零数据、零训练的异常监督，效果媲美传统方法。
- 适合希望降低标注成本的异常检测研究者与工业应用开发者。
语义分割网络在独立同分布数据假设下表现优异，但对未知语义类别的异常检测能力有限，因其训练时仅覆盖有限视觉概念。传统异常分割需在异常样本上微调，涉及额外的数据收集、标注和模型重训练。为避免此繁琐流程，本文提出将视觉语言（VL）编码器融入现有异常检测器，利用其广泛语义预训练提升对异常的感知能力。同时，设计一种新评分函数，通过文本提示实现无需数据和训练的异常监督。提出的VL4AD模型结合最大置信度提示集成与类别合并策略，在多个主流基准数据集上取得竞争力结果，验证了视觉语言模型在像素级异常检测中的潜力。
原文摘要 · Abstract (English)
Semantic segmentation networks have achieved significant success under the assumption of independent and identically distributed data. However, these networks often struggle to detect anomalies from unknown semantic classes due to the limited set of visual concepts they are typically trained on. To address this issue, anomaly segmentation often involves fine-tuning on outlier samples, necessitating additional efforts for data collection, labeling, and model retraining. Seeking to avoid this cumbersome work, we take a different approach and propose to incorporate Vision-Language (VL) encoders into existing anomaly detectors to leverage the semantically broad VL pre-training for improved outlier awareness. Additionally, we propose a new scoring function that enables data- and training-free outlier supervision via textual prompts. The resulting VL4AD model, which includes max-logit prompt ensembling and a class-merging strategy, achieves competitive performance on widely used benchmark datasets, thereby demonstrating the potential of vision-language models for pixel-wise anomaly detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。