arXiv:2412.13516cs.LG2024-12AAAI被引 7

从因果视角建模实例相关噪声,提升标签去噪准确性。

Learning Causal Transition Matrix for Instance-dependent Label Noise

  • 引入潜变量构建因果图,区分抗噪与敏感成分
  • 理论保证下逼近真实转移矩阵,提升清洁标签推断精度
  • 适合处理复杂场景下的实例相关噪声问题

噪声标签在机器学习中不可避免且有害,会因过拟合损害模型泛化能力。过渡矩阵在设计统计一致算法中至关重要,但通常不可识别。传统方法假设过渡矩阵为实例无关,即误标概率不依赖实例特征,这一假设在复杂现实场景中不成立。本文从因果视角研究噪声标签的数据生成过程,发现未观测的潜变量可能同时影响实例本身和标注过程,导致过渡矩阵难以识别。为此,我们提出统一的因果图模型,将输入实例分解为受潜变量影响的敏感成分和不受影响的抗噪成分,据此可识别出具有理论保证的‘因果过渡矩阵’,近似真实转移矩阵。基于此,我们设计了一种新型训练框架,显式建模该因果关系,从而更准确地推断清洁标签。

原文摘要 · Abstract (English)

Noisy labels are both inevitable and problematic in machine learning methods, as they negatively impact models' generalization ability by causing overfitting. In the context of learning with noise, the transition matrix plays a crucial role in the design of statistically consistent algorithms. However, the transition matrix is often considered unidentifiable. One strand of methods typically addresses this problem by assuming that the transition matrix is instance-independent; that is, the probability of mislabeling a particular instance is not influenced by its characteristics or attributes. This assumption is clearly invalid in complex real-world scenarios. To better understand the transition relationship and relax this assumption, we propose to study the data generation process of noisy labels from a causal perspective. We discover that an unobservable latent variable can affect either the instance itself, the label annotation procedure, or both, which complicates the identification of the transition matrix. To address various scenarios, we have unified these observations within a new causal graph. In this graph, the input instance is divided into a noise-resistant component and a noise-sensitive component based on whether they are affected by the latent variable. These two components contribute to identifying the ``causal transition matrix'', which approximates the true transition matrix with theoretical guarantee. In line with this, we have designed a novel training framework that explicitly models this causal relationship and, as a result, achieves a more accurate model for inferring the clean label.

因果学习噪声标签过渡矩阵去噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。