用主动学习与对比学习,仅用5%标注数据实现生物图像分类新纪录。
Co-Training with Active Contrastive Learning and Meta-Pseudo-Labeling on 2D Projections for Deep Semi-Supervised Learning
- 通过双网络协同训练,结合伪标签与主动采样优化模型。
- 在仅5%标注数据下超越6种先进方法,3%数据时性能相当。
- 适合标注成本高、数据量大但标签稀缺的生物图像场景。
深度学习模型训练受限于精确标注数据的稀缺性,尤其在需耗时费力标注的领域。半监督学习(SSL)利用少量标注数据和大量未标注数据应对此问题,但现有顶尖方法通常依赖预训练特征和大型验证集,且标签样本常随机选取,忽视信息量。本文提出active-DeepFA,融合对比学习(CL)、基于教师-学生结构的元伪标签和主动学习(AL),在无预训练的CNN架构上实现图像分类。该方法将DeepFA引入协同训练框架,采用两个互协网络缓解伪标签带来的确认偏差。初始阶段以少量标注样本进行监督对比学习预热;随后定期对网络深层特征的2D投影进行标签传播,交叉交换最可靠的伪标签,并主动标注最有意义的新样本加入标注集。各网络独立最小化包含监督对比、监督与半监督损失的综合目标函数,提升表征能力。在三个挑战性生物图像数据集上仅使用5%标注数据即超越基线并优于六种前沿方法;且仅需3%标注数据即可达到类似性能,显著降低标注负担。
原文摘要 · Abstract (English)
A major challenge that prevents the training of DL models is the limited availability of accurately labeled data. This shortcoming is highlighted in areas where data annotation becomes a time-consuming and error-prone task. In this regard, SSL tackles this challenge by capitalizing on scarce labeled and abundant unlabeled data; however, SoTA methods typically depend on pre-trained features and large validation sets to learn effective representations for classification tasks. In addition, the reduced set of labeled data is often randomly sampled, neglecting the selection of more informative samples. Here, we present active-DeepFA, a method that effectively combines CL, teacher-student-based meta-pseudo-labeling and AL to train non-pretrained CNN architectures for image classification in scenarios of scarcity of labeled and abundance of unlabeled data. It integrates DeepFA into a co-training setup that implements two cooperative networks to mitigate confirmation bias from pseudo-labels. The method starts with a reduced set of labeled samples by warming up the networks with supervised CL. Afterward and at regular epoch intervals, label propagation is performed on the 2D projections of the networks' deep features. Next, the most reliable pseudo-labels are exchanged between networks in a cross-training fashion, while the most meaningful samples are annotated and added into the labeled set. The networks independently minimize an objective loss function comprising supervised contrastive, supervised and semi-supervised loss components, enhancing the representations towards image classification. Our approach is evaluated on three challenging biological image datasets using only 5% of labeled samples, improving baselines and outperforming six other SoTA methods. In addition, it reduces annotation effort by achieving comparable results to those of its counterparts with only 3% of labeled data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。