arXiv:2409.19151cs.CL2024-09ICLR被引 40

用一本语法书教大模型翻译冷门语言,发现平行例句比语法解释更重要。

Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?

  • 用语法书中的平行例句提升翻译性能,而非依赖语法说明。
  • 在卡拉马昂语、尼泊尔语等低资源语言上达到接近专业模型的翻译水平。
  • 适合做低资源语言翻译或语法研究的学者参考,强调数据类型匹配任务。

极度低资源(XLR)语言缺乏大规模语料库,促使利用词典和语法书等可用资源。机器翻译仅靠一本书(Tanzer et al., 2024)提出,通过提示长上下文大模型使用一本语法书,即可实现英语-卡拉马昂语翻译,这是语言学助力NLP任务的典型案例。我们探究该能力来源,发现几乎所有性能提升均来自书中的平行例句,而非语法解释。在尼泊尔语和瓜拉尼语上也得到类似结果。仅通过微调一个编码器-解码器模型,即可达到与使用语法书的大模型相当的性能。进一步测试了语法判断和词义标注任务,发现引入类型学特征提示可在这些更相关任务中取得领先效果。因此强调:对XLR语言,翻译应优先收集平行数据,语言学任务则需语法数据。我们未发现长上下文大模型能有效利用语法解释进行翻译,故建议多语言低资源翻译任务应聚焦平行数据而非语言描述。

原文摘要 · Abstract (English)

Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from One Book (Tanzer et al., 2024) suggests that prompting long-context LLMs with one grammar book enables English-Kalamang translation, an XLR language unseen by LLMs - a noteworthy case of linguistics helping an NLP task. We investigate the source of this translation ability, finding almost all improvements stem from the book's parallel examples rather than its grammatical explanations. We find similar results for Nepali and Guarani, seen low-resource languages, and we achieve performance comparable to an LLM with a grammar book by simply fine-tuning an encoder-decoder translation model. We then investigate where grammar books help by testing two linguistic tasks, grammaticality judgment and gloss prediction, and we explore what kind of grammatical knowledge helps by introducing a typological feature prompt that achieves leading results on these more relevant tasks. We thus emphasise the importance of task-appropriate data for XLR languages: parallel examples for translation, and grammatical data for linguistic tasks. As we find no evidence that long-context LLMs can make effective use of grammatical explanations for XLR translation, we conclude data collection for multilingual XLR tasks such as translation is best focused on parallel data over linguistic description.

低资源翻译语法书大模型平行数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。