对比不同模型在开放世界下的未知样本识别与增量学习能力。
Open Horizons: Evaluating Deep Models in the Wild
- 用线性探测和多种打分函数评估视觉编码器对未知类的区分能力。
- CLIP在未知样本检测中表现最佳，ConCM在10次少样本学习中达84.7%准确率。
- 揭示主干网络与打分机制影响，适合关注开放世界部署的研究者。
开放世界部署要求模型既能识别已知类别，又能在新类别出现时保持可靠。我们在CIFAR-10上对开放集识别（OSR）和少样本类增量学习（FSCIL）进行了统一实验。对于OSR，比较了三个预训练冻结视觉编码器：ResNet-50、ConvNeXt-Tiny和CLIP ViT-B/16，采用线性探测及MSP、Energy、Mahalanobis和kNN四种后处理打分函数。在AUROC、AUPR、FPR@95和OSCR等指标下，CLIP始终表现出最强的已知与未知样本可分性，其中Energy在各主干中性能最稳定。对于FSCIL，使用部分冻结的ResNet-50，在1、5、10次样本设置下对比改进版SPPR、OrCo和ConCM。ConCM在10次设置下达到84.7%准确率，混淆矩阵最清晰，且所有方法在超过5次后均出现性能饱和。受控评估揭示了主干结构与评分机制对未知检测的影响，以及基于原型的方法如何缓解增量适应中的灾难性遗忘。
原文摘要 · Abstract (English)
Open-world deployment requires models to recognize both known categories and remain reliable when novel classes appear. We present a unified experimental study spanning open-set recognition (OSR) and few-shot class-incremental learning (FSCIL) on CIFAR-10. For OSR, we compare three pretrained frozen visual encoders: ResNet-50, ConvNeXt-Tiny and CLIP ViT-B/16,using a linear probe and four post-hoc scoring functions, namely MSP, Energy, Mahalanobis and kNN. Across metrics,such as, AUROC, AUPR, FPR@95, and OSCR, CLIP consistently yields the strongest separability between known and unknown samples, with Energy providing the most stable performance across backbones. For FSCIL, we compare modified SPPR, OrCo, and ConCM using partially frozen ResNet-50 across 1-, 5-, and 10-shot scenarios. ConCM achieves 84.7% accuracy in the 10-shot setting with the cleanest confusion matrix, while all methods show saturation beyond 5 shots. Our controlled evaluation reveals how the backbone architecture and scoring mechanisms affect unknown detection and how prototype-based methods mitigate catastrophic forgetting during incremental adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。