通过对称化交叉熵，提出新型抗噪损失函数，提升模型在错误标签下的鲁棒性。
Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels
- 将交叉熵对称化，导出多分类无铰链损失，满足抗噪理论条件。
- 该损失在等分值附近线性逼近任何对称损失，具基础性作用。
- 新设计的SGCE与alpha-MAE可灵活控制平滑度，实验表现优于现有方法。
标注训练数据成本高且易出错，设计对标签噪声鲁棒的损失函数至关重要。对称性条件能提供抗噪的理论保障。本文研究从任意多分类损失函数唯一分解中产生的对称化方法：对称化交叉熵得到线性扩展的多分类无铰链损失。不同于二分类情形，多分类版本需特定系数才能满足对称性。在合理假设下，证明该多分类无铰链损失是唯一的凸对称损失。同时发现其具有根本局部特性：任何对称损失在等分值得分向量附近的线性近似均等价于该无铰链损失。进一步提出SGCE与alpha-MAE，二者在无铰链损失与平均绝对误差间插值，并可调控β-平滑性。在标准含噪标签基准测试上，性能媲美现有鲁棒损失函数。
原文摘要 · Abstract (English)
Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。