无需训练即可同时检测逻辑与结构异常，利用大模型实现零样本异常检测。
Towards Training-free Anomaly Detection with Vision and Language Foundation Models
- 用大模型生成匹配建议，构建逻辑与结构异常的推理框架。
- 在多粒度层级上检测异常，包括像素块、兴趣区域和组合关系。
- 适合工业质检等需快速部署的零样本场景，性能超越有监督方法。
异常检测在工业质量检查等实际应用中至关重要。然而，现有方法多聚焦局部结构异常，忽视包含逻辑约束的组合异常。本文提出 LogSAD，一种无需训练的多模态框架，可统一处理逻辑与结构异常检测。首先，设计“思想匹配”架构，利用 GPT-4V 等先进多模态大模型生成匹配建议，构建异常检测所需的兴趣点与组合规则。其次，实现多粒度异常检测，涵盖像素块、兴趣集合及视觉与语言基础模型间的组合匹配。随后，引入校准模块对不同检测器的异常分数进行对齐，并采用集成策略作出最终判断。实验表明，该方法在无需训练的情况下，仍达到领先性能，甚至优于部分有监督方法，展现出极强鲁棒性与有效性。代码已开源：https://github.com/zhang0jhon/LogSAD。
原文摘要 · Abstract (English)
Anomaly detection is valuable for real-world applications, such as industrial quality inspection. However, most approaches focus on detecting local structural anomalies while neglecting compositional anomalies incorporating logical constraints. In this paper, we introduce LogSAD, a novel multi-modal framework that requires no training for both Logical and Structural Anomaly Detection. First, we propose a match-of-thought architecture that employs advanced large multi-modal models (i.e. GPT-4V) to generate matching proposals, formulating interests and compositional rules of thought for anomaly detection. Second, we elaborate on multi-granularity anomaly detection, consisting of patch tokens, sets of interests, and composition matching with vision and language foundation models. Subsequently, we present a calibration module to align anomaly scores from different detectors, followed by integration strategies for the final decision. Consequently, our approach addresses both logical and structural anomaly detection within a unified framework and achieves state-of-the-art results without the need for training, even when compared to supervised approaches, highlighting its robustness and effectiveness. Code is available at https://github.com/zhang0jhon/LogSAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。