通过多层提示与双分支推理,提升CLIP在零样本异常检测中的泛化能力。
GenCLIP: Generalizing CLIP Prompts for Zero-shot Anomaly Detection
- 采用多层视觉提示融合不同层级特征,增强提示鲁棒性
- 双分支推理平衡类别特异性与整体泛化性能
- 自适应过滤无关文本,确保语义对齐有效性
零样本异常检测(ZSAD)旨在利用CLIP的零样本能力,通过文本提示与视觉特征匹配来识别未见类别的异常。其核心挑战在于如何稳定地学习通用提示并有效利用,同时兼顾泛化性与类别特异性。现有工作虽探索过通用提示,但其稳定优化与有效部署仍具挑战。本文提出GenCLIP框架,通过多层提示与双分支推理更有效地学习和使用通用提示。多层提示整合了CLIP各层的类别特定视觉线索,丰富通用提示的表征能力。结合多层视觉特征,进一步增强泛化性能。为平衡特异性与泛化性,引入双分支推理:视觉增强分支捕捉细粒度类别特征,查询仅分支侧重泛化。两分支互补输出提升跨未见类别的异常检测稳定性与可靠性。此外,提出自适应文本提示过滤机制,剔除训练中未出现的无关或异常类别名称,确保仅有有意义的文本输入参与最终视觉-语言对齐。
原文摘要 · Abstract (English)
Zero-shot anomaly detection (ZSAD) aims to identify anomalies in unseen categories by leveraging CLIP's zero-shot capabilities to match text prompts with visual features. A key challenge in ZSAD is learning general prompts stably and utilizing them effectively, while maintaining both generalizability and category specificity. Although general prompts have been explored in prior works, achieving their stable optimization and effective deployment remains a significant challenge. In this work, we propose GenCLIP, a novel framework that learns and leverages general prompts more effectively through multi-layer prompting and dual-branch inference. Multi-layer prompting integrates category-specific visual cues from different CLIP layers, enriching general prompts with more comprehensive and robust feature representations. By combining general prompts with multi-layer visual features, our method further enhances its generalization capability. To balance specificity and generalization, we introduce a dual-branch inference strategy, where a vision-enhanced branch captures fine-grained category-specific features, while a query-only branch prioritizes generalization. The complementary outputs from both branches improve the stability and reliability of anomaly detection across unseen categories. Additionally, we propose an adaptive text prompt filtering mechanism, which removes irrelevant or atypical class names not encountered during CLIP's training, ensuring that only meaningful textual inputs contribute to the final vision-language alignment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。