用持续指令微调提升低资源语言翻译性能
Lius: Translation Model Based Instructional Lingustic Using Continual Instruction Tuning In Kupang Malay
- 基于双语词典设计指令，通过持续指令微调优化模型
- 在多项指标上比标准指令微调模型高4-6分，优于NMT和多语言大模型10-13分
- 适合低资源语言翻译研究者，减少对平行语料依赖
大型语言模型（LLMs）为翻译任务带来新可能，但在处理低资源语言时常出现性能下降。为解决此问题，本文提出一种针对低资源语言Kupang Malay的LLM微调方法。通过利用双语词典中的显式词汇与语义特征设计指令集，并引入持续指令微调（CIT）训练范式，实现迭代式指令驱动训练。实验表明，所提模型Lius在多个评估指标上显著优于标准指令微调模型，提升4-6分，同时超越神经机器翻译（NMT）和多语言大模型10-13分。结果表明该方法能有效缓解低资源语言翻译对大规模平行数据的依赖。
原文摘要 · Abstract (English)
Large Language Models (LLMs) offer new potential for translation tasks but often experience performance degradation when handling low-resource languages. To address this limitation, we propose an approach for fine-tuning LLMs on a low-resource language, Kupang Malay. Our approach involves designing a set of instructions by leveraging explicit lexical and semantic features from a bilingual dictionary, and introducing Continual Instruction Tuning (CIT), a training paradigm that enables iterative instruction-based training. Experimental results demonstrate that our model, named Lius, yields notable improvements over standard instruction-tuned models by outperforming 4-6 points, and surpassing both Neural Machine Translation (NMT) and Multilingual LLM models by 10-13 points on several evaluation metrics. These findings highlight the potential of our approach to mitigate the reliance on large-scale parallel data in low-resource language translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。