用贝叶斯方法学习动态提示流,提升零样本异常检测泛化能力
Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection
- 将提示空间建模为贝叶斯概率分布,实现可学习的提示生成
- 在15个工业与医疗数据集上均优于现有方法,尤其对未见类别表现更优
- 适合需要跨类别泛化、无标注异常数据的工业质检与医学影像场景
近期视觉语言模型(如CLIP)在零样本异常检测(ZSAD)中表现出色。通过利用辅助数据训练,这些模型可直接在目标数据集上进行跨类别异常检测,例如检测工业产品表面缺陷或器官组织中的肿瘤。现有方法通常通过人工设计或可学习提示向量构建文本提示,但面临三大挑战:1)手工提示需大量专家知识和试错;2)单一形式可学习提示难以捕捉复杂异常语义;3)无约束提示空间限制了对未见类别的泛化能力。为此,本文提出贝叶斯提示流学习(Bayes-PFL),从贝叶斯视角建模提示空间为可学习的概率分布。具体地,设计提示流模块以学习图像相关和图像无关的分布,联合正则化文本提示空间,提升模型在未见类别上的泛化能力。所学分布用于采样生成多样化提示,有效覆盖提示空间。此外,引入残差跨模型注意力(RCA)模块,更好地对齐动态文本嵌入与细粒度图像特征。在15个工业与医疗数据集上的大量实验表明,本方法性能显著优于现有方法。代码已开源。
原文摘要 · Abstract (English)
Recently, vision-language models (e.g. CLIP) have demonstrated remarkable performance in zero-shot anomaly detection (ZSAD). By leveraging auxiliary data during training, these models can directly perform cross-category anomaly detection on target datasets, such as detecting defects on industrial product surfaces or identifying tumors in organ tissues. Existing approaches typically construct text prompts through either manual design or the optimization of learnable prompt vectors. However, these methods face several challenges: 1) handcrafted prompts require extensive expert knowledge and trial-and-error; 2) single-form learnable prompts struggle to capture complex anomaly semantics; and 3) an unconstrained prompt space limits generalization to unseen categories. To address these issues, we propose Bayesian Prompt Flow Learning (Bayes-PFL), which models the prompt space as a learnable probability distribution from a Bayesian perspective. Specifically, a prompt flow module is designed to learn both image-specific and image-agnostic distributions, which are jointly utilized to regularize the text prompt space and improve the model's generalization on unseen categories. These learned distributions are then sampled to generate diverse text prompts, effectively covering the prompt space. Additionally, a residual cross-model attention (RCA) module is introduced to better align dynamic text embeddings with fine-grained image features. Extensive experiments on 15 industrial and medical datasets demonstrate our method's superior performance. The code is available at https://github.com/xiaozhen228/Bayes-PFL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。