提出新损失函数，让模型逼近贝叶斯最优分类精度。
Universal Training of Neural Networks to Achieve Bayes Optimal Classification Accuracy
- 用f-散度构造贝叶斯误差上界，可采样计算。
- 在多个数据集上训练效果优于或等同交叉熵。
- 适合追求泛化性能的分类任务研究者。
本文引入f-散度概念，为一般分类任务构建了贝叶斯误差率的新上界。该上界可通过参数化模型输出采样计算。基于此，我们提出贝叶斯最优学习阈值（BOLT）损失，其最小化可使分类模型逼近贝叶斯误差率。我们在图像与文本分类任务中验证该损失，涵盖MNIST、Fashion-MNIST、CIFAR-10和IMDb数据集。数值实验表明，使用BOLT训练的模型在挑战性数据集上表现优于或等同于交叉熵，凸显其提升泛化能力的潜力。
原文摘要 · Abstract (English)
This work invokes the notion of $f$-divergence to introduce a novel upper bound on the Bayes error rate of a general classification task. We show that the proposed bound can be computed by sampling from the output of a parameterized model. Using this practical interpretation, we introduce the Bayes optimal learning threshold (BOLT) loss whose minimization enforces a classification model to achieve the Bayes error rate. We validate the proposed loss for image and text classification tasks, considering MNIST, Fashion-MNIST, CIFAR-10, and IMDb datasets. Numerical experiments demonstrate that models trained with BOLT achieve performance on par with or exceeding that of cross-entropy, particularly on challenging datasets. This highlights the potential of BOLT in improving generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。