arXiv:2508.10556cs.CVcs.AI2025-08

用外部知识增强提示词,让模型更好识别分布外样本。

Retrieval-Augmented Prompt for OOD Detection

  • 通过检索外部文本知识,动态生成更丰富的异常提示词。
  • 1次提示下在ImageNet-1k上FPR95降低7.05%,AUROC提升1.71%。
  • 适合需要快速适应新异常场景的部署系统使用。

分布外(OOD)检测对机器学习模型在真实环境中的可靠部署至关重要,能准确识别与训练数据分布不同的测试样本。现有方法依赖辅助异常样本或分布内(ID)数据生成异常信息进行训练,但由于异常样本有限且与真实测试异常不匹配,难以提供充分的语义监督,导致性能不佳。为此,我们提出一种新型OOD检测方法——检索增强提示(Retrieval-Augmented Prompt, RAP)。RAP通过检索外部知识来增强预训练视觉-语言模型的提示词,为OOD检测提供更强的语义监督。训练时,基于与外部文本知识的联合相似性,检索出描述异常的词语以扩充模型的OOD提示;测试时,根据遇到的异常样本实时动态更新提示词,使模型能快速适应测试环境。大量实验表明,RAP在大规模OOD检测基准上达到当前最优性能。例如,在ImageNet-1k数据集上的1次提示设置下,平均FPR95降低7.05%,AUROC提升1.71%。此外,全面的消融实验验证了各模块的有效性及方法设计动机。

原文摘要 · Abstract (English)

Out-of-Distribution (OOD) detection is crucial for the reliable deployment of machine learning models in-the-wild, enabling accurate identification of test samples that differ from the training data distribution. Existing methods rely on auxiliary outlier samples or in-distribution (ID) data to generate outlier information for training, but due to limited outliers and their mismatch with real test OOD samples, they often fail to provide sufficient semantic supervision, leading to suboptimal performance. To address this, we propose a novel OOD detection method called Retrieval-Augmented Prompt (RAP). RAP augments a pre-trained vision-language model's prompts by retrieving external knowledge, offering enhanced semantic supervision for OOD detection. During training, RAP retrieves descriptive words for outliers based on joint similarity with external textual knowledge and uses them to augment the model's OOD prompts. During testing, RAP dynamically updates OOD prompts in real-time based on the encountered OOD samples, enabling the model to rapidly adapt to the test environment. Our extensive experiments demonstrate that RAP achieves state-of-the-art performance on large-scale OOD detection benchmarks. For example, in 1-shot OOD detection on the ImageNet-1k dataset, RAP reduces the average FPR95 by 7.05% and improves the AUROC by 1.71% compared to previous methods. Additionally, comprehensive ablation studies validate the effectiveness of each module and the underlying motivations of our approach.

OOD检测提示工程知识检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。