用少量标注数据+大量无标注数据，让大模型更高效地适配新任务。
Semi-supervised Fine-tuning for Large Language Models
- 通过双层知识传播，把标注数据的知识迁移到无标注数据中。
- 在7个数据集上提升模型表现，尤其在标注数据少时优势明显。
- 适合标注数据稀缺但有大量无标注数据的场景，如垂直领域应用。
监督微调（SFT）在将大语言模型（LLM）适配到特定领域或任务中至关重要。然而实际应用中可用的标注数据有限，严重制约了SFT的效果。为此，我们提出半监督微调（SemiFT）任务，并设计名为SemiEvol的框架，采用传播-选择机制实现LLM对齐。知识传播方面，SemiEvol采用双层方法，通过权重内和上下文内两种方式，将标注数据的知识传递至无标注数据。知识选择方面，引入协同学习机制，筛选出质量更高的伪响应样本。我们在GPT-4o-mini和Llama-3.1上对七个通用或领域特定数据集进行了实验，结果显示模型在目标数据上的性能显著提升。此外，与SFT和自进化方法对比，SemiEvol在混合数据场景中展现出更强的实用性。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is crucial in adapting large language model (LLMs) to a specific domain or task. However, only a limited amount of labeled data is available in practical applications, which poses a severe challenge for SFT in yielding satisfactory results. Therefore, a data-efficient framework that can fully exploit labeled and unlabeled data for LLM fine-tuning is highly anticipated.Towards this end, we introduce a semi-supervised fine-tuning(SemiFT) task and a framework named SemiEvol for LLM alignment from a propagate-and-select manner. For knowledge propagation, SemiEvol adopts a bi-level approach, propagating knowledge from labeled data to unlabeled data through both in-weight and in-context methods. For knowledge selection, SemiEvol incorporates a collaborative learning mechanism, selecting higher-quality pseudo-response samples. We conducted experiments using GPT-4o-mini and Llama-3.1 on seven general or domain-specific datasets, demonstrating significant improvements in model performance on target data. Furthermore, we compared SemiEvol with SFT and self-evolution methods, highlighting its practicality in hybrid data scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。