arXiv:2409.18316cs.LGstat.ML2024-09被引 2

提出TaMatch框架,缓解半监督学习中的确认偏见问题。

Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective

  • 基于先验分布与模型状态动态调整伪标签生成
  • 对不同类别伪标签加权,减少偏差类影响,增强少数类学习
  • 无需已知先验分布,适合真实场景,显著提升分类性能

半监督学习(SSL)常出现确认偏见,模型过度偏好某些类别,导致伪标签错误在自训练过程中累积。与有丰富静态数据分布的监督学习不同,SSL缺乏纠正此类自我强化偏差的机制,需在每一步训练中引入去偏干预。尽管去偏伪标签生成已被广泛研究,其有效利用仍不足。分析表明,应降低偏差类数据对参数更新的影响,同时加强对少数类的关注。为此,本文提出TaMatch,一个统一的去偏训练框架。TaMatch通过结合先验目标分布与模型学习状态,计算缩放比例,在每步训练中估计并修正偏差,调整未标记数据的原始预测以生成去偏伪标签。在利用阶段,根据预测类别对这些标签进行差异化加权,提升训练公平性,降低类别偏差。此外,TaMatch可动态调整目标分布,适应模型学习进度,适用于先验分布未知的实际场景。实验表明,TaMatch在多个具有挑战性的图像分类任务上显著优于现有最先进方法,凸显了伪标签去偏生成与有效利用在SSL中的关键作用。

原文摘要 · Abstract (English)

Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes, leading to errors in predicted pseudo labels that accumulate under a self-training paradigm. Unlike supervised settings, which benefit from a rich, static data distribution, SSL inherently lacks mechanisms to correct this self-reinforced bias, necessitating debiased interventions at each training step. Although the generation of debiased pseudo labels has been extensively studied, their effective utilization remains underexplored. Our analysis indicates that data from biased classes should have a reduced influence on parameter updates, while more attention should be given to underrepresented classes. To address these challenges, we introduce TaMatch, a unified framework for debiased training in SSL. TaMatch employs a scaling ratio derived from both a prior target distribution and the model's learning status to estimate and correct bias at each training step. This ratio adjusts the raw predictions on unlabeled data to produce debiased pseudo labels. In the utilization phase, these labels are differently weighted according to their predicted class, enhancing training equity and minimizing class bias. Additionally, TaMatch dynamically adjust the target distribution in response to the model's learning progress, facilitating robust handling of practical scenarios where the prior distribution is unknown. Empirical evaluations show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks, highlighting the critical importance of both the debiased generation and utilization of pseudo labels in SSL.

半监督学习去偏训练伪标签确认偏见

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。