提出高效元标签修正方法,显著降低噪声标签训练成本与时间
Efficient Bilevel Optimization for Meta Label Correction in Noisy Label Learning

- 采用一步内循环更新与动态屏障机制加速优化
- 在高噪声率下仍保持优异性能,训练时间大幅缩短
- 适合需要低成本标注的工业级图像分类任务
使用噪声标签训练深度神经网络可降低数据标注成本,但可能引入模型偏差。现有元标签修正方法通过额外的元模型在小规模干净数据上修正大规模噪声数据,但其元模型更新需计算主模型内层的超梯度,导致计算开销大。为提升效率,本文首次将动态屏障梯度下降引入标准元标签修正框架。虽能近似一阶复杂度,但缺乏抑制噪声信号泄漏与稳定元模型学习的机制。为此,提出EBOMLC方法,包含三方面改进:一步内循环更新、混合上界损失函数及对齐感知动态屏障。在CIFAR-10和CIFAR-100上的实验表明,该方法在高噪声率下持续优于多个基线,同时显著降低元标签修正的训练时间。
原文摘要 · Abstract (English)
Training a deep neural network with noisy labels could reduce data annotation cost but may introduce noise into the learned model. In meta label correction approaches, an additional meta model besides the main model is trained with a small, clean dataset to correct the large, noisy dataset. However, the update of the meta model requires the computation of hypergradients at the inner step of the main model which signif- icantly increases the computational cost. To improve the training efficiency, we first introduce the dynamic barrier gradient descent into standard meta label correction. While this naive extenstion is able to speed up the training process to approximately first- order complexity, it lacks mechanisms to prevent the leakage of noisy signals to the main model and to stabilize the learning of the meta model. Based on this observation, we propose the EBOMLC method, which is designed with three key improvements including one-step inner loop update, mixture upper loss and alignment- aware dynamic barrier. Empirical results on CIFAR-10 and CIFAR-100 demonstrate that EBOMLC consistently outperforms other baselines, especially under high noise rate settings, while reducing training time of the meta label correction approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。