用单卡训练让冷门语言翻译更准，结合开源模型与检索增强。
Language verY Rare for All
- 融合开源大模型微调、检索增强生成和高资源语言迁移。
- 单卡训练下实现法语-摩纳哥语双向翻译，效果超越或媲美顶尖模型。
- 适合资源稀缺语言研究者，尤其关注低成本部署的实践场景。
为克服语言障碍，如NLLB这样的编码器-解码器模型已将机器翻译扩展至稀有语言，部分模型（如NLLB 1.3B）甚至可在单张GPU上训练。尽管通用大模型在翻译中表现良好，但经过特定任务微调的开源大模型在未知语料任务中也表现出高度竞争力。本文提出LYRA（Language verY Rare for All），结合开源大模型微调、检索增强生成（RAG）以及相关高资源语言的迁移学习。本研究仅聚焦于单GPU训练，以促进实际应用。研究重点为法语与摩纳哥语之间的双向翻译，该语言因语料稀缺而未被现有工具支持。结果表明，LYRA在稀有语言翻译中表现优异，频繁超越并始终匹配当前最优编码器-解码器模型。
原文摘要 · Abstract (English)
In the quest to overcome language barriers, encoder-decoder models like NLLB have expanded machine translation to rare languages, with some models (e.g., NLLB 1.3B) even trainable on a single GPU. While general-purpose LLMs perform well in translation, open LLMs prove highly competitive when fine-tuned for specific tasks involving unknown corpora. We introduce LYRA (Language verY Rare for All), a novel approach that combines open LLM fine-tuning, retrieval-augmented generation (RAG), and transfer learning from related high-resource languages. This study is exclusively focused on single-GPU training to facilitate ease of adoption. Our study focuses on two-way translation between French and Monégasque, a rare language unsupported by existing translation tools due to limited corpus availability. Our results demonstrate LYRA's effectiveness, frequently surpassing and consistently matching state-of-the-art encoder-decoder models in rare language translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。