增加微调时的语言多样性，能显著提升翻译模型性能。
The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation
- 在132个翻译方向上系统测试语言多样性影响
- 语言多样性提升后，跨语言翻译质量显著改善
- 适合需要多语言支持的翻译系统开发者
先前研究对大语言模型微调中的语言多样性存在分歧：部分报告有益，另一些则未发现优势。通过在132个翻译方向上进行受控微调实验，我们系统性地解决了这些矛盾。结果表明，微调过程中扩展语言多样性，能提升无监督及出人意料的有监督翻译对的翻译质量，即使使用较少多样性的模型仅在这些有监督对上微调也未能达到同等效果。然而，当语言多样性超过某一阈值后，收益趋于饱和甚至下降。我们发现，更高的语言多样性促使模型生成更具语言无关性的表征，这种表征适应性可解释为何多样性增强带来性能提升。
原文摘要 · Abstract (English)
Prior research diverges on language diversity in LLM fine-tuning: Some studies report benefits while others find no advantages. Through controlled fine-tuning experiments across 132 translation directions, we systematically resolve these disparities. We find that expanding language diversity during fine-tuning improves translation quality for both unsupervised and -- surprisingly -- supervised pairs, despite less diverse models being fine-tuned exclusively on these supervised pairs. However, benefits plateau or decrease beyond a certain diversity threshold. We show that increased language diversity creates more language-agnostic representations. These representational adaptations help explain the improved performance in models fine-tuned with greater diversity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。