用大模型提升不完全标签学习效果,关键在选对模型并合理微调。
Tuning the Right Foundation Models is What you Need for Partial Label Learning
- 在不完全标签数据上,用大模型替代传统CNN可显著提效。
- 11个大模型在8个数据集上表现相近,说明现有方法差异不大。
- 文本嵌入初始化和零样本过滤能有效提升模型泛化能力。
不完全标签学习(PLL)旨在从标注不精确的数据中训练具有泛化能力的分类器,这在真实场景中十分常见。现有方法多通过训练卷积神经网络逐步修正和恢复真实标签,但对可迁移表征的大模型关注不足。本文对11个基础模型在13种PLL方法、8个基准数据集及3种PLL场景下进行了全面评估。提出一种高效微调框架PartialCLIP用于大模型在PLL中的应用。结果表明:1)使用大模型能带来显著性能提升;2)不同方法间表现高度相似;3)在不同模糊程度下保持稳定;4)性能严重依赖大模型选择与适配策略。此外,我们验证了基于零样本CLIP的文本嵌入初始化与候选标签过滤的有效性。实验分析揭示了当前方法的局限,为构建更通用的PLL模型提供了重要启示。代码见https://github.com/SEU-hk/PartialCLIP。
原文摘要 · Abstract (English)
Partial label learning (PLL) seeks to train generalizable classifiers from datasets with inexact supervision, a common challenge in real-world applications. Existing studies have developed numerous approaches to progressively refine and recover ground-truth labels by training convolutional neural networks. However, limited attention has been given to foundation models that offer transferrable representations. In this work, we empirically conduct comprehensive evaluations of 11 foundation models across 13 PLL approaches on 8 benchmark datasets under 3 PLL scenarios. We further propose PartialCLIP, an efficient fine-tuning framework for foundation models in PLL. Our findings reveal that current PLL approaches tend to 1) achieve significant performance gains when using foundation models, 2) exhibit remarkably similar performance to each other, 3) maintain stable performance across varying ambiguity levels, while 4) are susceptible to foundation model selection and adaptation strategies. Additionally, we demonstrate the efficacy of text-embedding classifier initialization and effective candidate label filtering using zero-shot CLIP. Our experimental results and analysis underscore the limitations of current PLL approaches and provide valuable insights for developing more generalizable PLL models. The source code can be found at https://github.com/SEU-hk/PartialCLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。