arXiv:2510.12209cs.LGcs.AI2025-10

提出新型标签噪声处理方法,理论揭示其三阶段训练机制。

Revisiting Meta-Learning with Noisy Labels: Reweighting Dynamics and Theoretical Guarantees

  • 基于小量干净样本引导,通过相似性加权耦合优化
  • 训练分三阶段:对齐、过滤、敏感期,噪声权重渐趋零
  • 设计轻量替代方案,避免复杂双层优化,性能更稳定

带噪声标签的学习仍具挑战性,因过参数化网络会记忆错误监督信号。基于元学习的样本重加权方法利用少量干净样本指导训练,但其行为与训练动态缺乏理论解释。本文对标签噪声下的元重加权进行了严格理论分析,发现其训练轨迹分为三个阶段:(i) 对齐阶段放大与干净子集一致的样本,抑制冲突样本;(ii) 过滤阶段使噪声样本权重趋近零,直到干净子集损失趋于平稳;(iii) 后过滤阶段噪声过滤变得对扰动敏感。其机制是训练信号与干净子集信号间的相似性加权耦合,以及干净子集损失收缩;当干净子集损失足够小时,耦合项消失,元重加权失去区分能力。基于此分析,我们提出一种轻量级替代方案,整合均值中心化、行偏移和标签符号调制,实现更稳定的性能,同时避免昂贵的双层优化。在合成与真实噪声标签基准上,该方法持续优于强基线重加权/选择方法。

原文摘要 · Abstract (English)

Learning with noisy labels remains challenging because over-parameterized networks memorize corrupted supervision. Meta-learning-based sample reweighting mitigates this by using a small clean subset to guide training, yet its behavior and training dynamics lack theoretical understanding. We provide a rigorous theoretical analysis of meta-reweighting under label noise and show that its training trajectory unfolds in three phases: (i) an alignment phase that amplifies examples consistent with a clean subset and suppresses conflicting ones; (ii) a filtering phase driving noisy example weights toward zero until the clean subset loss plateaus; and (iii) a post-filtering phase in which noise filtration becomes perturbation-sensitive. The mechanism is a similarity-weighted coupling between training and clean subset signals together with clean subset training loss contraction; in the post-filtering regime where the clean-subset loss is sufficiently small, the coupling term vanishes and meta-reweighting loses discriminatory power. Guided by this analysis, we propose a lightweight surrogate for meta-reweighting that integrates mean-centering, row shifting, and label-signed modulation, yielding more stable performance while avoiding expensive bi-level optimization. Across synthetic and real noisy-label benchmarks, our method consistently outperforms strong reweighting/selection baselines.

元学习噪声标签重加权

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。