arXiv:2509.16516cs.LG2025-09EMNLP被引 4

用大模型指导的协同训练,让小模型在无标签数据上表现更优。

LLM-Guided Co-Training for Text Classification

  • 大模型为无标签数据生成标签,两个网络互换重要性权重协同训练
  • 在5个基准数据集上4个达顶尖水平,14种方法中排名第一
  • 适合缺乏标注数据但有大模型可用的场景

本文提出一种由大语言模型(LLM)引导的加权协同训练方法。在该方法中,使用LLM对无标签数据生成的标签作为目标标签,两个基于编码器的网络通过多轮迭代相互训练:首先,将所有样本输入两个网络,并记录每个网络对自身预测置信度的历史估计;其次,根据每个网络对特定样本的LLM标签质量信念,动态计算每个样本的重要性权重;最后,两个网络交换重要性权重——每个网络以对方网络提供的权重对所有样本进行反向传播并更新参数。通过战略性利用LLM生成的指导信息,该方法显著优于传统自监督学习方法,尤其在无标签数据丰富的场景下表现突出。实验结果表明,在5个基准数据集中有4个达到最先进水平,且在14种对比方法中根据弗里德曼检验排名第一。研究揭示了半监督学习的新方向——大模型作为知识放大器,使主干协同训练模型能高效实现顶尖性能。

原文摘要 · Abstract (English)

In this paper, we introduce a novel weighted co-training approach that is guided by Large Language Models (LLMs). Namely, in our co-training approach, we use LLM labels on unlabeled data as target labels and co-train two encoder-only based networks that train each other over multiple iterations: first, all samples are forwarded through each network and historical estimates of each network's confidence in the LLM label are recorded; second, a dynamic importance weight is derived for each sample according to each network's belief in the quality of the LLM label for that sample; finally, the two networks exchange importance weights with each other -- each network back-propagates all samples weighted with the importance weights coming from its peer network and updates its own parameters. By strategically utilizing LLM-generated guidance, our approach significantly outperforms conventional SSL methods, particularly in settings with abundant unlabeled data. Empirical results show that it achieves state-of-the-art performance on 4 out of 5 benchmark datasets and ranks first among 14 compared methods according to the Friedman test. Our results highlight a new direction in semi-supervised learning -- where LLMs serve as knowledge amplifiers, enabling backbone co-training models to achieve state-of-the-art performance efficiently.

半监督学习大模型引导协同训练文本分类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。