用人工验证的合成数据,为资源匮乏的那加克里奥尔语构建了首个完整NLP工具包。
NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data
- 借助大模型生成、母语者人工校验,构建高质量合成数据
- 训练出93.81%准确率的词性标注模型,远超零样本基线
- 适合低资源语言研究者,提供可复现的数据构建框架
全球绝大多数语言,尤其是那加克里奥尔语等克里奥尔语,严重缺乏自然语言处理资源,阻碍其在数字技术中的应用。本文提出NagaNLP,一个开源工具包,通过一种新型方法——基于大模型生成但经人工验证的合成数据——实现自举。我们设计了一个多阶段流程:由专家引导的大模型(Gemini)生成候选语料,再由母语者进行精炼与标注。该混合方法产出10,000对对话数据及高质量标注语料,支撑基础任务。为评估效果,我们训练了判别式与生成式模型。微调后的XLM-RoBERTa-base模型在词性标注上达到93.81%准确率(F1-Macro 0.90),命名实体识别达F1-Macro 0.75,显著优于强零样本基线。此外,微调的Llama-3.2-3B Instruct模型(命名为NagaLLaMA)在对话任务中表现优异,困惑度降至3.85,较少样本版本(96.76)提升一个数量级。我们发布NagaNLP工具包,包含所有数据集、模型与代码,为此前未被充分服务的语言提供基础资源,并为其他低资源场景提供可复现的数据减贫方案。
原文摘要 · Abstract (English)
The vast majority of the world's languages, particularly creoles like Nagamese, remain severely under-resourced in Natural Language Processing (NLP), creating a significant barrier to their representation in digital technology. This paper introduces NagaNLP, a comprehensive open-source toolkit for Nagamese, bootstrapped through a novel methodology that relies on LLM-driven but human-validated synthetic data generation. We detail a multi-stage pipeline where an expert-guided LLM (Gemini) generates a candidate corpus, which is then refined and annotated by native speakers. This synthetic-hybrid approach yielded a 10K pair conversational dataset and a high-quality annotated corpus for foundational tasks. To assess the effectiveness of our methodology, we trained both discriminative and generative models. Our fine-tuned XLM-RoBERTa-base model establishes a new benchmark for Nagamese, achieving a 93.81\% accuracy (0.90 F1-Macro) on Part-of-Speech tagging and a 0.75 F1-Macro on Named Entity Recognition, massively outperforming strong zero-shot baselines. Furthermore, we fine-tuned a Llama-3.2-3B Instruct model, named NagaLLaMA, which demonstrates superior performance on conversational tasks, achieving a Perplexity of 3.85, an order of magnitude improvement over its few-shot counterpart (96.76). We release the NagaNLP toolkit, including all datasets, models, and code, providing a foundational resource for a previously underserved language and a reproducible framework for reducing data scarcity in other low-resource contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。