arXiv:2411.17113cs.LG2024-11NeurIPS被引 7

用鲁棒优化方法提升噪声标签下的模型性能,尤其适合标注质量参差的场景。

Learning from Noisy Labels via Conditional Distributionally Robust Optimization

  • 基于条件分布鲁棒优化框架,构建抗干扰的标签估计机制。
  • 在合成与真实数据集上均显著优于现有方法,噪声环境下准确率提升明显。
  • 适合高噪声标注场景,如众包数据、医学图像等专业领域应用。

众包标注虽为大规模数据标注提供实用方案,但因标注者能力不一导致标签噪声严重,影响模型学习效果。现有方法多通过估计给定样本与噪声标注下的真实标签后验来推断真标签或调整损失函数,但常忽略该后验可能存在的模型误设问题,尤其在高噪声场景下性能下降明显。本文提出基于条件分布鲁棒优化(CDRO)的学习框架,将问题建模为以参考分布为中心、基于距离的模糊集内最坏风险最小化。通过强对偶性分析,推导出最坏风险的上界,并获得每条数据点的对偶鲁棒风险解析解。进而设计一种新型鲁棒伪标签算法,利用似然比检验构建伪经验分布,作为CDRO中的稳健参考概率分布。此外,推导出经验鲁棒风险及对偶问题最优拉格朗日乘子的闭式表达,实现鲁棒性与模型拟合间的合理权衡。实验在合成与真实数据集上验证了方法优越性。

原文摘要 · Abstract (English)

While crowdsourcing has emerged as a practical solution for labeling large datasets, it presents a significant challenge in learning accurate models due to noisy labels from annotators with varying levels of expertise. Existing methods typically estimate the true label posterior, conditioned on the instance and noisy annotations, to infer true labels or adjust loss functions. These estimates, however, often overlook potential misspecification in the true label posterior, which can degrade model performances, especially in high-noise scenarios. To address this issue, we investigate learning from noisy annotations with an estimated true label posterior through the framework of conditional distributionally robust optimization (CDRO). We propose formulating the problem as minimizing the worst-case risk within a distance-based ambiguity set centered around a reference distribution. By examining the strong duality of the formulation, we derive upper bounds for the worst-case risk and develop an analytical solution for the dual robust risk for each data point. This leads to a novel robust pseudo-labeling algorithm that leverages the likelihood ratio test to construct a pseudo-empirical distribution, providing a robust reference probability distribution in CDRO. Moreover, to devise an efficient algorithm for CDRO, we derive a closed-form expression for the empirical robust risk and the optimal Lagrange multiplier of the dual problem, facilitating a principled balance between robustness and model fitting. Our experimental results on both synthetic and real-world datasets demonstrate the superiority of our method.

噪声标签鲁棒优化伪标签众包标注

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。