构建缅甸语自然语言推理数据集,探索低资源语言的模型适配方案。
Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar
- 通过众包+专家校验构建缅甸语推理数据集myXNLI。
- 数据增强使缅甸语模型准确率提升2个百分点。
- 方法对其他低资源语言也有效,适合多语言NLP研究者。
尽管自然语言处理(NLP)取得显著进展,将大语言模型(LLM)应用于低资源语言仍是重大挑战。以跨语言自然语言推理(XNLI)为例,该任务在15种语言上评估NLP系统的跨语言能力。本文针对一种新增的低资源语言——缅甸语,扩展了XNLI任务,并做出三项核心贡献:首先,采用社区众包与专家验证相结合的方式构建名为myXNLI的缅甸语数据集,通过分析证明专家校验对提升众包数据质量至关重要;其次,评估了多种多语言模型在myXNLI上的表现,并探索数据增强方法,结果表明该方法使缅甸语模型准确率最高提升2个百分点,同时提升了其他语言性能;第三,检验了这些数据增强方法在XNLI中其他低资源语言上的泛化能力。相关数据集已公开供研究使用。
原文摘要 · Abstract (English)
Despite dramatic recent progress in NLP, it is still a major challenge to apply Large Language Models (LLM) to low-resource languages. This is made visible in benchmarks such as Cross-Lingual Natural Language Inference (XNLI), a key task that demonstrates cross-lingual capabilities of NLP systems across a set of 15 languages. In this paper, we extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages, and make three core contributions. First, we build a dataset called Myanmar XNLI (myXNLI) using community crowd-sourced methods, as an extension to the existing XNLI corpus. This involves a two-stage process of community-based construction followed by expert verification; through an analysis, we demonstrate and quantify the value of the expert verification stage in the context of community-based construction for low-resource languages. We make the myXNLI dataset available to the community for future research. Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance. Our data-augmentation methods improve model accuracy by up to 2 percentage points for Myanmar, while uplifting other languages at the same time. Third, we investigate how well these data-augmentation methods generalise to other low-resource languages in the XNLI dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。