arXiv:2502.11671cs.CLcs.AI2025-02ACL被引 19

用大模型生成多样语义的文本增强数据,提升模型泛化能力。

Diversity-oriented Data Augmentation with Large Language Models

  • 训练大模型作为多样性重述器,生成语义多样的数据
  • 在12个真实数据集上平均性能提升10.52%,超越基线超3个百分点
  • 适合需要提升模型鲁棒性的自然语言任务研究者

数据增强是自然语言处理中通过生成多样化样本丰富训练数据的关键技术,对提升模型的鲁棒性和泛化能力至关重要。然而,当前方法普遍关注样本数量增加而忽视样本分布多样性,易导致模型过拟合。为此,本文提出一种面向多样性的数据增强框架(DoAug)。该框架采用多样性导向的微调策略,训练大语言模型作为多样化的文本重述器,能够生成多样化的改写样本。随后,在精选出的高信息量核心子集上应用该重述器,将生成的改写句与原始数据融合,构建更具多样性的增强数据集。在12个真实文本数据集上的实验表明,所提出的微调大模型增强器在保持标签一致性的同时显著提升数据多样性,从而增强下游任务的鲁棒性与性能。具体而言,平均性能提升达10.52%,优于次优基线超过三个百分点。

原文摘要 · Abstract (English)

Data augmentation is an essential technique in natural language processing (NLP) for enriching training datasets by generating diverse samples. This process is crucial for improving the robustness and generalization capabilities of NLP models. However, a significant challenge remains: \textit{Insufficient Attention to Sample Distribution Diversity}. Most existing methods focus on increasing the sample numbers while neglecting the sample distribution diversity, which can lead to model overfitting. In response, we explore data augmentation's impact on dataset diversity and propose a \textbf{\underline{D}}iversity-\textbf{\underline{o}}riented data \textbf{\underline{Aug}}mentation framework (\textbf{DoAug}). % \(\mathscr{DoAug}\) Specifically, we utilize a diversity-oriented fine-tuning approach to train an LLM as a diverse paraphraser, which is capable of augmenting textual datasets by generating diversified paraphrases. Then, we apply the LLM paraphraser to a selected coreset of highly informative samples and integrate the paraphrases with the original data to create a more diverse augmented dataset. Finally, we conduct extensive experiments on 12 real-world textual datasets. The results show that our fine-tuned LLM augmenter improves diversity while preserving label consistency, thereby enhancing the robustness and performance of downstream tasks. Specifically, it achieves an average performance gain of \(10.52\%\), surpassing the runner-up baseline with more than three percentage points.

数据增强大模型多样性NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。