用大模型理解测试图像,自适应生成负向文本空间提升异常检测效果
ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
- 利用多模态大模型分析测试图像,生成精准描述异常分布的负面文本
- 在ImageNet上将FPR95降低3.1%,显著优于现有方法
- 无需训练、零样本部署,适合开放环境下的实时异常检测
引入负向标签(NLs)已被证明能有效提升分布外(OOD)检测性能。然而,现有方法缺乏对OOD图像的理解能力,难以构建准确的负向空间;同时,缺少与正常类别语义相近的负向标签,限制了近似OOD检测能力。为此,我们提出通过多模态大语言模型(MLLM)的推理能力,自适应构建负向文本空间(ANTS)。具体而言,从历史测试图像中缓存可能为OOD的样本,通过提示MLLM描述这些图像,生成能精确刻画异常分布的表达性负面句子,增强远距离OOD检测。对于视觉上接近正常类别的近似OOD样本,缓存与其视觉相似的正常类别子集,并利用MLLM推理生成针对该子集的视觉相似负向标签,有效减少误报。设计自适应加权评分机制,平衡两类负向文本空间,使方法在近/远OOD任务间灵活切换。在ImageNet基准上,我们的方法将FPR95降低3.1%,达到新最优。此外,本方法无需训练、支持零样本部署,具备高可扩展性。代码已开源。
原文摘要 · Abstract (English)
The introduction of negative labels (NLs) has proven effective in enhancing Out-of-Distribution (OOD) detection. However, existing methods often lack an understanding of OOD images, making it difficult to construct an accurate negative space. Furthermore, the absence of negative labels semantically similar to ID labels constrains their capability in near-OOD detection. To address these issues, we propose shaping an Adaptive Negative Textual Space (ANTS) by leveraging the understanding and reasoning capabilities of multimodal large language models (MLLMs). Specifically, we cache images likely to be OOD samples from the historical test images and prompt the MLLM to describe these images, generating expressive negative sentences that precisely characterize the OOD distribution and enhance far-OOD detection. For the near-OOD setting, where OOD samples resemble the in-distribution (ID) subset, we cache the subset of ID classes that are visually similar to historical test images and then leverage MLLM reasoning to generate visually similar negative labels tailored to this subset, effectively reducing false negatives and improving near-OOD detection. To balance these two types of negative textual spaces, we design an adaptive weighted score that enables the method to handle different OOD task settings (near-OOD and far-OOD), making it highly adaptable in open environments. On the ImageNet benchmark, our ANTS significantly reduces the FPR95 by 3.1\%, establishing a new state-of-the-art. Furthermore, our method is training-free and zero-shot, enabling high scalability. Codes are available at https://github.com/ZhuWenjie98/ANTS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。