模型在未见细粒度类别下不仅准确率下降，还变得过度自信。
Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift
- 通过系统测试发现，模型在未见子类型下准确率下降但置信度不降。
- 相同准确率损失下，图像退化导致置信度大幅下降，说明问题非普遍性。
- 仅用已见子类型校准无法完全修复问题，需结合校准评估鲁棒性。
细粒度子类型鲁棒性关注的是：当测试样本来自训练中未出现的细分类别（但仍属于已知粗粒度类别）时，模型是否仍能保持正确的粗粒度预测。以往研究几乎仅关注准确率。本文首次系统研究了这一问题在ImageNet、BREEDS、iNaturalist和CIFAR-100上的表现，涵盖五种架构。结果发现，面对未见子类型时，模型准确率下降，但置信度几乎不变，导致系统性过度自信。在相同准确率损失下，通用图像退化引起的置信度下降远大于子类型变化，说明该现象并非准确率下降的普遍后果。模型对可见退化敏感，却对类别内新异不敏感。在已见子类型上进行校准可缩小差距，但无法消除。分布外检测分数也仅弱提示异常输入。因此，子类型鲁棒性应通过校准而非准确率单独评估。
原文摘要 · Abstract (English)
Subtype robustness asks whether a model keeps the correct coarse prediction when test examples come from fine-grained subtypes absent from training but still inside a known coarse category. Prior work studies this almost entirely through accuracy. We ask whether the model also stays calibrated. We present the first systematic study of the question across ImageNet, BREEDS, iNaturalist and CIFAR-100 with five architectures. Calibration breaks down on unseen subtypes, where accuracy drops while confidence barely follows, leaving the model systematically overconfident exactly where it has become less accurate. At matched accuracy loss, generic image corruption causes a much larger drop in confidence, so the effect is not a general consequence of losing accuracy. The model reacts to visible degradation but not to in-taxonomy novelty. Recalibration tuned on seen subtypes narrows the gap but does not close it, and out-of-distribution scores flag the affected inputs only weakly. Subtype robustness should therefore be evaluated through calibration, not accuracy alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。