数据增强效果因任务而异,不能靠大模型生成质量判断。
When Does Data Augmentation Help? Evaluating LLM and Back-Translation Methods for Hausa and Fongbe NLP
- 用大模型生成和反向翻译两种方法增强西非语言数据
- 命名实体识别任务中两种方法都未提升,甚至降低性能
- 词性标注任务中各有增益,说明需按任务设计增强策略
低资源非洲语言的自然语言处理受限于数据稀缺。本文评估了基于大模型生成(Gemini 2.5 Flash)和反向翻译(NLLB-200)两种数据增强方法在豪萨语和丰贝语上的表现,使用MasakhaNER 2.0和MasakhaPOS基准进行命名实体识别(NER)与词性标注(POS)任务测试。结果表明,增强效果取决于任务类型而非语言或大模型生成质量。对于NER任务,两种方法均未提升,豪萨语NER下降0.24% F1,丰贝语下降1.81% F1;对于POS任务,大模型增强使丰贝语准确率提升0.33%,反向翻译使豪萨语提升0.17%,但反向翻译使丰贝语下降0.35%,对豪萨语影响不显著。同一批合成数据在丰贝语上对NER有害而对POS有益,说明任务结构比合成数据质量更关键。研究挑战了‘大模型生成质量决定增强效果’的假设,提出应将数据增强视为任务特异性干预,而非通用预处理步骤。
原文摘要 · Abstract (English)
Data scarcity limits NLP development for low-resource African languages. We evaluate two data augmentation methods -- LLM-based generation (Gemini 2.5 Flash) and back-translation (NLLB-200) -- for Hausa and Fongbe, two West African languages that differ substantially in LLM generation quality. We assess augmentation on named entity recognition (NER) and part-of-speech (POS) tagging using MasakhaNER 2.0 and MasakhaPOS benchmarks. Our results reveal that augmentation effectiveness depends on task type rather than language or LLM quality alone. For NER, neither method improves over baseline for either language; LLM augmentation reduces Hausa NER by 0.24% F1 and Fongbe NER by 1.81% F1. For POS tagging, LLM augmentation improves Fongbe by 0.33% accuracy, while back-translation improves Hausa by 0.17%; back-translation reduces Fongbe POS by 0.35% and has negligible effect on Hausa POS. The same LLM-generated synthetic data produces opposite effects across tasks for Fongbe -- hurting NER while helping POS -- suggesting task structure governs augmentation outcomes more than synthetic data quality. These findings challenge the assumption that LLM generation quality predicts augmentation success, and provide actionable guidance: data augmentation should be treated as a task-specific intervention rather than a universally beneficial preprocessing step.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。