用预训练模型生成噪声标签,实现高效下游适应。
Bridging Weakly-Supervised Learning and VLM Distillation: Noisy Partial Label Learning for Efficient Downstream Adaptation
- 双网络协同伪标签净化,提升噪声标签质量。
- 在多个数据集上达到优于传统方法的准确率,尤其在低标注成本下表现突出。
- 适合资源受限场景下的弱监督学习与模型蒸馏应用。
在噪声部分标签学习(NPLL)中,每个训练样本由多位噪声标注者提供一组候选标签。随着CLIP、LLaVA和GPT-4V等高性能预训练视觉语言模型(VLM)的出现,利用这些模型替代耗时的人工标注,实现无标注训练成为有前景的研究方向。本文研究了由预训练VLM生成的噪声部分标签的学习问题,提出一种协同一致性正则化(Co-Reg)框架。不同于传统方法假设的对称噪声,VLM生成的噪声具有实例依赖性,反映预训练模型的固有偏差,带来更大挑战。为此,我们联合训练两个神经网络,通过协同伪标签机制进行标签净化,并在标签与特征表示空间同时施加一致性正则化。此外,引入多种抗过拟合策略,包括对比表示与伪标签的交替优化,以及在共享特征空间中维护类别原型。该方法可进一步融合少量人工标注标签以提升性能。在多种设置下的大量实验验证了方法的有效性,凸显了将弱监督学习融入预训练模型知识蒸馏的潜力。
原文摘要 · Abstract (English)
In the context of noisy partial label learning (NPLL), each training sample is associated with a set of candidate labels annotated by multiple noisy annotators. With the emergence of high-performance pre-trained vision-language models (VLMs) such as CLIP, LLaVA, and GPT-4V, leveraging these models to replace time-consuming manual annotation and enable annotation-free training has become a promising research direction. This paper studies learning from noisy partial labels generated by pre-trained VLMs and proposes a collaborative consistency regularization (Co-Reg) framework. Unlike symmetric noise commonly assumed in traditional noisy label learning, VLM-generated noise is instance-dependent and reflects the intrinsic biases of pre-trained models, posing greater challenges. To address this issue, we jointly train two neural networks to perform collaborative label purification via a co-pseudo-labeling mechanism, while enforcing consistency regularization in both label and feature representation spaces. In addition, multiple anti-overfitting strategies are introduced, including alternating optimization of contrastive representations and pseudo-labels, as well as maintaining class prototypes in a shared feature space. The proposed method can further incorporate few-shot manually annotated labels for performance enhancement. Extensive experiments under various settings demonstrate the effectiveness of our approach and highlight the potential of integrating weakly supervised learning into the knowledge distillation of pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。