arXiv:2608.12340cs.CL2026-08

对比发现:生成文本增广不如保留类别结构的检索方法有效。

Class-Structure Preservation Beats Diversity: A Comprehensive Benchmark of Text Augmentation Methods for Imbalanced Text Classification

论文配图:Class-Structure Preservation Beats Diversity: A Comprehensive Benchmark of Text Augmentation Methods for Imbalanced Text Classification
图 1 · 摘自论文原文
  • 用7个数据集、11种方法做对照实验,涵盖经典扰动、嵌入检索和大模型生成。
  • 大模型增广在极端不平衡下性能比嵌入检索差0.063分(宏F1),差距随不平衡加剧。
  • 表面多样性无用,反而因文本过长和标签均匀化损害分类效果,应优先保结构。

随着大语言模型的发展,生成式数据增强在自然语言处理的不平衡文本分类中受到关注。然而,目前尚无实证基准将基于LLM的增广与经典的嵌入空间SMOTE式检索(EmbSMOTE)进行对比。本研究在涵盖2至28个类、不平衡比达1.1至500以上的7个公开文本分类数据集上,构建了包含11种方法的受控基准,评估方式包括宏F1、Welch's t检验、五项分布度量及基于Qwen3-8B的LLM敏感性分析,每组实验运行5次随机种子。结果表明,所有基于LLM的方法在统计上等价或劣于EmbSMOTE,且性能差距随不平衡程度增加而单调扩大,在GoEmotions-28上达到ΔF1_macro≈0.063。进一步发现,表面独特性与下游性能无关,而大模型特有的文本延长和标签分布均匀化现象则与准确率负相关。这说明有效变量并非表面多样性,而是类条件结构保真度——即增广样本对训练分布中类条件几何结构的保持程度。因此,对于不平衡多分类任务,应默认采用基于检索的过采样,部署大模型增广前需设立更高实证门槛。

原文摘要 · Abstract (English)

With the rapid advancement of large language models (LLMs), generative data augmentation has attracted considerable attention for imbalanced text classification in natural language processing. However, no empirical benchmark to date has compared LLM-based augmentation against the embedding-space SMOTE-style retrieval (EmbSMOTE), a strong classical reference for imbalanced classification. In this study, a controlled benchmark of 11 augmentation methods, spanning classical perturbation, embedding-space retrieval, and LLM-based generation, is newly constructed on seven public text classification datasets covering class counts $K=2$-$28$ and imbalance ratios of 1.1 to over 500, evaluated with five random seeds per cell using macro F1, Welch's $t$-tests, five distributional metrics, and an LLM-family sensitivity analysis based on Qwen3-8B. The experimental results reveal that all LLM-based methods are statistically equivalent or inferior to EmbSMOTE, with the performance gap widening monotonically as imbalance increases and reaching $Δ\text{F1}_\text{macro}\!\approx\!0.063$ on GoEmotions-28. Furthermore, it is observed that surface-level uniqueness has negligible correlation with downstream performance, whereas LLM-specific artifacts, such as text elongation and label-distribution uniformization, are negatively associated with classification accuracy. Compared with six LLM-based and four classical augmentation baselines, these results demonstrate that the effective variable is not surface-level diversity but class-conditional structural fidelity, namely the degree to which augmented samples preserve the class-conditioned geometry of the training distribution. Accordingly, retrieval-based oversampling should be adopted as the default for imbalanced multi-class classification, and a higher empirical bar should be required before LLM-based augmentation is deployed in practice.

文本增广不平衡分类大模型结构保真

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。