用多种相关语言嵌入均值初始化,提升低资源语言翻译效果
Embedding Initialization for Unseen Low-resource Languages in Multilingual NMT: A Case Study on Limbum-English Translation

- 用多个语言嵌入平均值作为新语言的初始词向量
- 在8837句对上达到46.7的chrF2++,接近最佳单语言代理
- 适合研究低资源非洲语言翻译,无需手动选代理语言
多语言神经机器翻译模型如NLLB-200覆盖200种语言,但仍有数千种语言未被支持,包括喀麦隆大部分格雷夫菲尔德班图语。当微调这些模型以处理未见语言时,需选择一个代理语言标记,但目前尚无系统性方法。本文提出一种嵌入初始化策略:将目标语言标记初始化为模型中多个类型学相关的已有语言嵌入的平均值。我们在基于新约文本的8,837句对平行语料和双语词典上评估了利姆布姆语到英语的翻译任务。对比模型包括:NLLB-200零样本(chrF2++ = 12.5)、从头训练的Transformer(chrF2++ = 14.5)、使用斯瓦希里语代理的NLLB-200微调(chrF2++ = 47.3),以及本文提出的平均嵌入初始化方法(chrF2++ = 46.7)。结果表明,多语言初始化性能与最优单语言代理相当,且两种NLLB-200变体均比从头训练基线提升超过32点chrF2++。这说明多语言迁移是极低资源班图语翻译中的主导因素,同时避免了启发式代理选择。然而,所有系统均未能保留声调标记,凸显仍存挑战。我们公开了数据集与代码以支持后续研究。
原文摘要 · Abstract (English)
Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon. When fine-tuning these models for an unseen language, practitioners must choose a proxy language token, yet no principled method exists for this selection. We implemented an embedding initialization strategy where a language token is the average of embeddings from multiple typologically related languages already in the mod el. We evaluate this approach on Limbum-to-English translation using a parallel corpus of 8,837 sentence pairs from New Testament text and a bilingual dictionary. We compare models: NLLB-200 zero-shot (chrF2++ = 12.5), a Transformer trained from scratch (chrF2++ = 14.5), NLLB-200 fine-tuned with a Swahili proxy token (chrF2++ = 47.3), and NLLB-200 with our averaged embedding initialization (chrF2++ = 46.7). We find that the multi-language initialization achieves performance comparable to the best single-language proxy. Both NLLB-200 variants improve over the from-scratch baseline by over 32 chrF2++ points. These results show that multilingual transfer is the dominant factor in extremely low-resource Bantu translation while eliminating the need for heuristic proxy selection. However, all systems fail to preserve tonal diacritics, highlighting an open challenge. We make our dataset and code available to support further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。