arXiv:2605.30140cs.CV2026-05

用大模型推理能力实现零样本异常检测,无需训练即可识别复杂异常。

AnomalyAgent: Training-Free Agentic Models for Zero-/Few-Shot Anomaly Detection

  • 基于多模态大模型构建智能体,通过工具集实现自适应异常推理。
  • 在零样本和少样本设置下性能显著优于现有方法,尤其擅长逻辑类异常检测。
  • 适合制造业、物流等领域需理解上下文的复杂异常场景应用。

得益于视觉语言模型(如CLIP)的泛化能力,许多零样本/少样本异常检测(AD)方法在多个数据集上取得了优异性能。然而,这些方法通常需要在大规模辅助数据集上进行大量训练以适配视觉语言模型,并且推理主要依赖视觉-文本嵌入相似性得分,缺乏对需深度上下文理解的复杂异常的推理能力。为此,我们提出 extbf{AnomalyAgent},一种无需训练的智能体框架,利用多模态大语言模型(MLLMs)的先进推理与泛化能力进行异常检测。核心包括: extbf{1)} 一套全面的以异常为中心的工具集,支持零样本环境下自适应的MLLM驱动式智能体推理; extbf{2)} 一个定制的记忆模块,通过少量上下文参考样例来锚定异常推理。我们将评估扩展至更丰富的异常类型,涵盖物流与制造场景中的逻辑/上下文异常,而不仅限于表面缺陷(如裂纹、凹痕)或明显病灶。大量实验表明,AnomalyAgent在零样本与少样本设置下均显著优于现有的无训练视觉语言模型方法及通用智能体方法,凸显其出色的泛化能力。代码可在该地址获取。

原文摘要 · Abstract (English)

Benefiting from generalizability of vision-language models (VLMs) such as CLIP, many zero-/few-shot anomaly detection (AD) approaches have achieved impressive detection performance across various datasets. Nevertheless, they require substantial training on large auxiliary datasets to adapt VLMs to anomaly detection, and their inference largely relies on visual-text embedding similarity-based anomaly scores, lacking reasoning abilities to detect complex anomalies that require in-depth contextual understanding. To address this limitation, we propose \textbf{AnomalyAgent}, a novel training-free, agentic framework that leverages the advanced reasoning and generalization capabilities of multimodal large language models (MLLMs) for anomaly detection. The key ingredients include \textbf{1)} a comprehensive anomaly-centric toolset that enables adaptive MLLM-driven, agentic anomaly reasoning in zero-shot settings, and \textbf{2)} a customized memory module that grounds anomaly reasoning with few-shot, in-context reference examples. We extend evaluation beyond the detection of simple anomalies (e.g., surface defects like cracks and dents and clear lesions) in widely used benchmarks to more diverse types of anomalies such as logical/contextual anomalies in logistics and manufacturing settings. Extensive experiment results demonstrate that our AnomalyAgent achieves substantially better performance compared to training-free VLM-based AD and generic agentic methods, highlighting its superior generalization capability in both zero-shot and few-shot anomaly detection settings. The code implementation can be find at this address.

异常检测大模型零样本智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。