用多个模型组合提升未知图像检测效果，比单一模型更准更稳。
COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP
- 组合闭域分类器、CLIP零样本分类器和特征线性探测器形成集成
- 在CIFAR100和ImageNet上达到最新最好性能，抗干扰能力更强
- 无需重新训练，可后处理部署，适合实际应用中的可靠视觉系统
分布外（OOD）检测是可信图像识别系统的重要组成部分，因为测试时可能出现未见类别。现有方法多基于单一分类器，导致研究分野：传统监督设置（如在CIFAR100上训练的ResNet18）与零样本设置（将类别名作为提示输入CLIP）。两者均受限于分类器在分布内（ID）数据上的表现。本文提出COOkeD，通过构建异构集成模型实现显著提升：结合端到端训练的闭域分类器、零样本的CLIP分类器及基于CLIP图像特征的线性探测器。虽看似复杂，但该方法模块化、可后处理，且利用预训练视觉语言模型，相比训练单一标准分类器开销极小。我们在CIFAR100和ImageNet等基准上评估，也涵盖更具挑战性的现实场景，包括训练标签噪声、测试时协变量偏移，以及此前被忽视的零样本偏移。尽管结构简单，COOkeD在性能与鲁棒性上均超越经典与基于CLIP的检测方法。代码已公开于https://github.com/glhr/COOkeD。
原文摘要 · Abstract (English)
Out-of-distribution (OOD) detection is an important building block in trustworthy image recognition systems as unknown classes may arise at test-time. OOD detection methods typically revolve around a single classifier, leading to a split in the research field between the classical supervised setting (e.g. ResNet18 classifier trained on CIFAR100) vs. the zero-shot setting (class names fed as prompts to CLIP). In both cases, an overarching challenge is that the OOD detection performance is implicitly constrained by the classifier's capabilities on in-distribution (ID) data. In this work, we show that given a little open-mindedness from both ends, remarkable OOD detection can be achieved by instead creating a heterogeneous ensemble - COOkeD combines the predictions of a closed-world classifier trained end-to-end on a specific dataset, a zero-shot CLIP classifier, and a linear probe classifier trained on CLIP image features. While bulky at first sight, this approach is modular, post-hoc and leverages the availability of pre-trained VLMs, thus introduces little overhead compared to training a single standard classifier. We evaluate COOkeD on popular CIFAR100 and ImageNet benchmarks, but also consider more challenging, realistic settings ranging from training-time label noise, to test-time covariate shift, to zero-shot shift which has been previously overlooked. Despite its simplicity, COOkeD achieves state-of-the-art performance and greater robustness compared to both classical and CLIP-based OOD detection methods. Code is available at https://github.com/glhr/COOkeD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。