arXiv:2507.10225cs.CV2025-07ICCV被引 8

用生成模型合成贴近分布边界的异常样本,提升图像识别的边界判别能力。

Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection

  • 通过多模态大模型引导迭代修复,生成贴近真实数据分布边界的异常图像。
  • 在ImageNet上实现当前最优的异常检测效果,参数和运行时间几乎无增加。
  • 适合需要高精度异常检测的视觉系统开发者,尤其关注边界样本的场景。

预训练的视觉语言模型在检测分布外(OOD)样本方面表现出色,但某些接近分布内(InD)数据特征空间边界的挑战性样本仍可能导致误分类。扩散模型和多模态大语言模型(MLLMs)等基础模型为此提供了潜在解决方案。本文提出SynOOD,利用基础模型生成合成的、具有挑战性的OOD数据,用于微调CLIP模型,从而增强对分布内外边界的判别能力。该方法采用由MLLM提供上下文提示的迭代修复过程,生成细粒度、与边界对齐的OOD样本,并基于能量分数等OOD评分梯度调整噪声,有效从分布边界采样。通过这些精心合成的图像,我们微调了CLIP图像编码器及文本编码器生成的负标签特征,强化了近边界OOD样本与一组负标签之间的关联。最终,SynOOD在大规模ImageNet基准上达到当前最优性能,且参数和运行时间增加极少。代码已开源:https://github.com/Jarvisgivemeasuit/SynOOD。

原文摘要 · Abstract (English)

Pre-trained vision-language models have exhibited remarkable abilities in detecting out-of-distribution (OOD) samples. However, some challenging OOD samples, which lie close to in-distribution (InD) data in image feature space, can still lead to misclassification. The emergence of foundation models like diffusion models and multimodal large language models (MLLMs) offers a potential solution to this issue. In this work, we propose SynOOD, a novel approach that harnesses foundation models to generate synthetic, challenging OOD data for fine-tuning CLIP models, thereby enhancing boundary-level discrimination between InD and OOD samples. Our method uses an iterative in-painting process guided by contextual prompts from MLLMs to produce nuanced, boundary-aligned OOD samples. These samples are refined through noise adjustments based on gradients from OOD scores like the energy score, effectively sampling from the InD/OOD boundary. With these carefully synthesized images, we fine-tune the CLIP image encoder and negative label features derived from the text encoder to strengthen connections between near-boundary OOD samples and a set of negative labels. Finally, SynOOD achieves state-of-the-art performance on the large-scale ImageNet benchmark, with minimal increases in parameters and runtime. Our approach significantly surpasses existing methods, and the code is available at https://github.com/Jarvisgivemeasuit/SynOOD.

异常检测生成模型视觉语言模型边界增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。