用图像级监督让大模型自己推理出异常位置,无需像素标注。
Reasoning-Driven Anomaly Detection and Localization with Image-Level Supervision
- 从大模型的推理过程提取异常关键词,生成像素级异常图。
- 通过强化学习对齐推理与视觉注意力,定位更准。
- 仅需图像标签就达到像素级检测效果,适合无标注场景。
多模态大语言模型(MLLM)在异常检测中展现出强大的推理与感知能力。然而,现有方法大多局限于图像级异常检测和文本推理,而像素级定位仍依赖外部视觉模块和密集标注。本文提出一种名为ReAL的方法,仅通过图像级监督,激活MLLM内在推理能力,实现异常检测、像素级定位与可解释推理,无需任何辅助组件或像素级标签。具体而言,ReAL从自回归推理过程中提取与异常相关的词元,并聚合其注意力响应生成像素级异常图。进一步引入一致性引导推理优化(CGRO)模块,利用强化学习使推理词元与视觉注意力对齐,提升推理连贯性与定位精度。在四个公开基准上的大量实验表明,该方法显著提升了异常检测、定位与可解释性。值得注意的是,尽管仅依赖图像级监督,其性能已媲美基于密集像素级监督训练的MLLM方法。代码已开源:https://github.com/YizhouJin313/ReADL。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) have recently demonstrated remarkable reasoning and perceptual abilities for anomaly detection. However, most approaches remain confined to image-level anomaly detection and textual reasoning, while pixel-level localization still relies on external vision modules and dense annotations. In this work, we activate the intrinsic reasoning potential of MLLMs to perform anomaly detection, pixel-level localization, and interpretable reasoning solely from image-level supervision, without any auxiliary components or pixel-wise labels. Specifically, we propose Reasoning-Driven Anomaly Localization (ReAL), which extracts anomaly-related tokens from the autoregressive reasoning process and aggregates their attention responses to produce pixel-level anomaly maps. We further introduce a Consistency-Guided Reasoning Optimization (CGRO) module that leverages reinforcement learning to align reasoning tokens with visual attentions, resulting in more coherent reasoning and accurate anomaly localization. Extensive experiments on four public benchmarks demonstrate that our method significantly improves anomaly detection, localization, and interpretability. Remarkably, despite relying solely on image-level supervision, our approach achieves performance competitive with MLLM-based methods trained under dense pixel-level supervision. Code is available at https://github.com/YizhouJin313/ReADL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。