提出新型损失函数MGCE，兼顾优化效率与抗噪声能力。
Minimax Generalized Cross-Entropy
- 采用极小极大框架重构GCE，实现分类边距的凸优化
- 在含标签噪声数据上准确率更高、收敛更快、校准性更好
- 适合对鲁棒性要求高的复杂分类任务
损失函数在监督分类中至关重要。交叉熵（CE）广泛应用，而平均绝对误差（MAE）虽具鲁棒性但难优化。广义交叉熵（GCE）介于两者之间，但现有形式在分类边距上导致非凸优化，易过拟合，复杂数据表现差。本文提出极小极大广义交叉熵（MGCE），使分类边距优化变为凸问题，并证明其可提供分类误差的上界。该双层凸优化可通过隐式微分计算随机梯度高效实现。在基准数据集上，MGCE在标签噪声下仍保持高准确率、快速收敛和良好校准性。
原文摘要 · Abstract (English)
Loss functions play a central role in supervised classification. Cross-entropy (CE) is widely used, whereas the mean absolute error (MAE) loss can offer robustness but is difficult to optimize. Interpolating between the CE and MAE losses, generalized cross-entropy (GCE) has recently been introduced to provide a trade-off between optimization difficulty and robustness. Existing formulations of GCE result in a non-convex optimization over classification margins that is prone to underfitting, leading to poor performances with complex datasets. In this paper, we propose a minimax formulation of generalized cross-entropy (MGCE) that results in a convex optimization over classification margins. Moreover, we show that MGCEs can provide an upper bound on the classification error. The proposed bilevel convex optimization can be efficiently implemented using stochastic gradient computed via implicit differentiation. Using benchmark datasets, we show that MGCE achieves strong accuracy, faster convergence, and better calibration, especially in the presence of label noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。