改进标签平滑，让模型更懂类别间关系。
Label Smoothing++: Enhanced Label Regularization for Training Neural Networks
- 对非目标类别分配不等权重，建模类别间关系
- 减少模型过自信预测，提升泛化能力
- 适合需要强泛化性的分类任务
使用独热编码标签训练神经网络常导致模型过自信和过拟合。标签平滑通过向独热标签添加均匀概率向量来实现正则化，但其对所有非目标类别赋予相等重要性，破坏了类别间的内在关系。本文提出一种新型标签正则化策略 Label Smoothing++，为非目标类别分配非零概率，并考虑其相互关系。该方法固定目标类别的标签，同时让网络学习非目标类别的标签表示。在多个数据集上的大量实验表明，Label Smoothing++ 能有效缓解过自信预测，促进类别间关系建模，提升模型泛化性能。
原文摘要 · Abstract (English)
Training neural networks with one-hot target labels often results in overconfidence and overfitting. Label smoothing addresses this issue by perturbing the one-hot target labels by adding a uniform probability vector to create a regularized label. Although label smoothing improves the network's generalization ability, it assigns equal importance to all the non-target classes, which destroys the inter-class relationships. In this paper, we propose a novel label regularization training strategy called Label Smoothing++, which assigns non-zero probabilities to non-target classes and accounts for their inter-class relationships. Our approach uses a fixed label for the target class while enabling the network to learn the labels associated with non-target classes. Through extensive experiments on multiple datasets, we demonstrate how Label Smoothing++ mitigates overconfident predictions while promoting inter-class relationships and generalization capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。