用模型自身预测构建概率化标签，提升分类准确率与校准性。
Deep Probabilistic Supervision for Image Classification
- 基于模型自身预测进行统计推断，生成样本专属的概率目标分布。
- 在ImageNet上使DenseNet-264准确率提升2.0%，CIFAR-100上校准误差降低40%。
- 适用于对抗标签噪声场景，尤其适合追求模型可信度与鲁棒性的研究者。
图像分类中深度神经网络的监督训练通常依赖硬标签，易导致模型过自信，影响校准性、泛化能力与鲁棒性。自蒸馏方法虽利用模型自身预测中的类别间与样本特异性信息以缓解此问题，但仍常依赖硬标签且未显式建模预测不确定性。为此，我们提出深度概率监督（Deep Probabilistic Supervision, DPS），一种基于统计推断构建样本专属目标分布的学习框架，初始化后完全脱离硬标签。实验表明，DPS在测试准确率上持续优于现有自蒸馏方法（如DenseNet-264在ImageNet上提升2.0%），且显著降低期望校准误差（ResNet-50在CIFAR-100上减少40%）。结合对比损失时，DPS在标签噪声下达到最先进的鲁棒性表现。
原文摘要 · Abstract (English)
Supervised training of deep neural networks for classification typically relies on hard targets, which promote overconfidence and can limit calibration, generalization, and robustness. Self-distillation methods aim to mitigate this by leveraging inter-class and sample-specific information present in the model's own predictions, but often remain dependent on hard targets without explicitly modeling predictive uncertainty. With this in mind, we propose Deep Probabilistic Supervision (DPS), a principled learning framework constructing sample-specific target distributions via statistical inference on the model's own predictions, remaining independent of hard targets after initialization. We show that DPS consistently yields higher test accuracy (e.g., +2.0% for DenseNet-264 on ImageNet) and significantly lower Expected Calibration Error (ECE) (-40% ResNet-50, CIFAR-100) than existing self-distillation methods. When combined with a contrastive loss, DPS achieves state-of-the-art robustness under label noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。