一种自适应噪声鲁棒损失函数,无需调参即可稳定训练。
Introducing Fractional Classification Loss for Robust Learning with Noisy Labels
- 用分数阶导数构建动态调节的损失函数
- 在多个数据集上实现最优性能且无需人工调参
- 适合标签噪声严重但希望免调参的场景
面对标签噪声时,鲁棒损失函数对深度神经网络训练至关重要,但现有方法需大量依赖数据集的超参数调优。本文提出分数分类损失(FCL),一种可在训练中自动校准鲁棒性的自适应损失函数。基于主动-被动损失框架,FCL以交叉熵损失的分数阶导数为活跃项,均方绝对误差(MAE)为被动项。通过调整分数阶导数阶数μ,可实现从类MAE鲁棒性到类交叉熵快速收敛的连续过渡。进一步将μ作为可学习参数嵌入梯度优化,自动调节鲁棒性与收敛速度的权衡。我们发现,降低对困难或误标样本的对数惩罚虽增强鲁棒性,却提高对简单或正确样本的惩罚,从而降低模型对其置信度。因此FCL能动态重构损失景观,在标签噪声下实现高效分类。大量实验表明,FCL在基准数据集上表现达到当前最优,且无需手动调参。
原文摘要 · Abstract (English)
Robust loss functions are crucial for training deep neural networks in the presence of label noise, yet existing approaches require extensive, dataset-specific hyperparameter tuning. In this work, we introduce Fractional Classification Loss (FCL), an adaptive robust loss that automatically calibrates its robustness to label noise during training. Built within the active-passive loss framework, FCL employs the fractional derivative of the Cross-Entropy (CE) loss as its active component and the Mean Absolute Error (MAE) as its passive loss component. With this formulation, we demonstrate that the fractional derivative order $μ$ spans a family of loss functions that interpolate between MAE-like robustness and CE-like fast convergence. Furthermore, we integrate $μ$ into the gradient-based optimization as a learnable parameter and automatically adjust it to optimize the trade-off between robustness and convergence speed. We reveal that FCL's unique property establishes a critical trade-off that enables the stable learning of $μ$: lower log penalties on difficult or mislabeled examples improve robustness but impose higher penalties on easy or clean data, reducing model confidence in them. Consequently, FCL can dynamically reshape its loss landscape to achieve effective classification performance under label noise. Extensive experiments on benchmark datasets show that FCL achieves state-of-the-art results without the need for manual hyperparameter tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。