用生成模型补足少样本类别，提升分类任务效果
The Synthetic Imputation Approach: Generating Optimal Synthetic Texts For Underrepresented Categories In Supervised Classification Tasks
- 用GPT-4o根据5个原始样本生成新文本，保持语义一致
- 75个原样本时性能媲美完整数据集，50个时过拟合可控
- 适合数据不平衡的实证研究，尤其文本分类任务
编码器-解码器大语言模型（如BERT、RoBERTa）在训练时要求所有类别都有充足样本以实现最佳性能。但在构建高质量训练集时，常难以获取足够多的样本覆盖所有类别。本文提出合成填补方法，利用生成式大模型GPT-4o，基于从样本中随机有放回抽取的5个原始例句，通过精心设计的提示生成合成文本。该方法确保新生成文本与原始文本差异显著，降低过拟合风险，同时保留核心语义，以最大化泛化性能。当原始样本达75个及以上时，合成填补的性能可媲美完整原始数据集；在50个样本下，过拟合仍处于低水平且可预测、可纠正。该方法为生成式LLM在研究中的应用开辟了新路径，帮助实证研究者优化数据分布，提升模型表现。
原文摘要 · Abstract (English)
Encoder-decoder Large Language Models (LLMs), such as BERT and RoBERTa, require that all categories in an annotation task be sufficiently represented in the training data for optimal performance. However, it is often difficult to find sufficient examples for all categories in a task when building a high-quality training set. In this article, I describe this problem and propose a solution, the synthetic imputation approach. Leveraging a generative LLM (GPT-4o), this approach generates synthetic texts based on careful prompting and five original examples drawn randomly with replacement from the sample. This approach ensures that new synthetic texts are sufficiently different from the original texts to reduce overfitting, but retain the underlying substantive meaning of the examples to maximize out-of-sample performance. With 75 original examples or more, synthetic imputation's performance is on par with a full sample of original texts, and overfitting remains low, predictable and correctable with 50 original samples. The synthetic imputation approach provides a novel role for generative LLMs in research and allows applied researchers to balance their datasets for best performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。