合成数据位置比数量更重要,能显著提升语用功能分类效果
The Impact of Synthetic Data Augmentation on Discourse-Pragmatic Function Classification

- 按真实数据分布生成合成样本,控制其在嵌入空间的距离
- 靠近真实数据的合成样本使宏平均F1提升0.113,准确率达74.8%
- 适合低资源语用分析任务,对模型决策边界有关键影响
合成数据增强已成为缓解NLP中类别不平衡的常见策略,但多数方法关注生成样本的数量与多样性,而非其与真实训练数据的几何关系。本文研究话语语用功能分类中的这一问题,该任务的数据稀疏性是结构性特征而非收集偏差所致。基于英国国家语料库中410个手工标注的英语词'look'实例,涵盖四种功能:注意力信号、指令、话语标记和感叹。使用Llama 3.1生成合成样本,并根据其在RoBERTa嵌入空间中与真实数据的余弦距离进行划分。比较六种训练条件,区别在于合成样本相对于经验决策边界的放置位置,保持增广数量一致。所有增广条件均优于仅使用真实数据的基线,其中核心邻近样本(NEAR)使宏平均F1提升最大(+0.113),而距离均衡混合方案达到最高准确率(0.748)。无任何条件提升AUC,表明增强主要改变决策边界而非改善模型概率估计。结果说明,在表示空间中合成样本的位置与数量同等重要,对低资源语用分类具有广泛意义。
原文摘要 · Abstract (English)
Synthetic data augmentation has become a common strategy for addressing class imbalance in NLP, but most approaches focus on the quantity and diversity of generated examples rather than their geometric relationship to real training data. We investigate this question in the context of discourse pragmatic function classification, a task where data sparsity is a structural feature rather than a collection artefact. Using 410 manually annotated instances of the English word look drawn from the British National Corpus, spanning four functions: Attention Signal, Directive, Discourse Marker, and Interjection. We generate synthetic training examples with Llama 3.1 and partition them by their cosine distance from real training data in RoBERTa embedding space. We compare six training conditions that differ in the placement of synthetic examples relative to the empirical decision boundary, while holding augmentation quantity constant across conditions. All augmented conditions improve macro F and accuracy over the real only baseline, but core proximal examples (NEAR) yield the largest gains in macro F (0.113), while a distance balanced mix achieves the highest accuracy (0.748). No condition improves AUC, indicating that augmentation shifts the decision boundary rather than improving the model's underlying probability estimates. These findings suggest that where synthetic examples land in representation space matters as much as how many are generated, with implications for low resource pragmatic classification more broadly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。