不用文字提示，纯视觉模型也能精准识别异常，还更稳定。
VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer
- 用可学习的视觉标记直接编码正常与异常概念
- 在13个基准上达到当前最优效果，工业与医疗场景均适用
- 无需文本分支，适配多种预训练视觉模型
零样本异常检测（ZSAD）需在无目标类别异常样本的情况下识别并定位异常。主流方法依赖视觉语言模型（如CLIP），通过手工或学习的提示集构建正常与异常语义，再计算图像-文本相似度实现开放集判别。然而该范式依赖文本编码器与跨模态对齐，易引发训练不稳定和参数冗余。本文重新审视文本分支在ZSAD中的必要性，提出纯视觉框架VisualAD，基于冻结的Vision Transformer骨干网络。引入两个可学习标记，分别编码正常性和异常性，通过多层自注意力机制与图像块交互，逐步获取高层语义，并引导块关注异常线索。此外，融合空间感知交叉注意力（SCA）模块和轻量级自对齐函数（SAF）：SCA注入细粒度空间信息，SAF在异常评分前重校准块特征。VisualAD在涵盖工业与医学领域的13个零样本异常检测基准上达到领先性能，且可无缝适配CLIP图像编码器、DINOv2等预训练视觉骨干。代码已开源。
原文摘要 · Abstract (English)
Zero-shot anomaly detection (ZSAD) requires detecting and localizing anomalies without access to target-class anomaly samples. Mainstream methods rely on vision-language models (VLMs) such as CLIP: they build hand-crafted or learned prompt sets for normal and abnormal semantics, then compute image-text similarities for open-set discrimination. While effective, this paradigm depends on a text encoder and cross-modal alignment, which can lead to training instability and parameter redundancy. This work revisits the necessity of the text branch in ZSAD and presents VisualAD, a purely visual framework built on Vision Transformers. We introduce two learnable tokens within a frozen backbone to directly encode normality and abnormality. Through multi-layer self-attention, these tokens interact with patch tokens, gradually acquiring high-level notions of normality and anomaly while guiding patches to highlight anomaly-related cues. Additionally, we incorporate a Spatial-Aware Cross-Attention (SCA) module and a lightweight Self-Alignment Function (SAF): SCA injects fine-grained spatial information into the tokens, and SAF recalibrates patch features before anomaly scoring. VisualAD achieves state-of-the-art performance on 13 zero-shot anomaly detection benchmarks spanning industrial and medical domains, and adapts seamlessly to pretrained vision backbones such as the CLIP image encoder and DINOv2. Code: https://github.com/7HHHHH/VisualAD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。