X-ALMA提升50种语言翻译质量，尤其改善低资源语言表现。
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale
- 通过插件式模块设计避免多语言训练冲突
- 在FLORES-200与WMT'23上全面超越Aya系列模型
- 新优化方法ARPO提升翻译偏好学习效果
大型语言模型（LLMs）在英语相关NLP任务中表现卓越，但多数基于英语预训练且缺乏多语言数据支持，导致中低资源语言翻译质量显著下降。本文提出X-ALMA，旨在实现50种不同语言的顶级翻译性能，无论资源丰富与否。该模型在FLORES-200和WMT'23测试集上，使用COMET-22评估，所有翻译方向均优于开源前沿模型Aya-101和Aya-23。其成功源于可插拔的语言特异性模块架构，有效防止训练中的语言冲突；结合创新训练策略与新型优化方法。最终阶段引入自适应拒选偏好优化（ARPO），在翻译任务中超越现有偏好优化方法。
原文摘要 · Abstract (English)
Large language models (LLMs) have achieved remarkable success across various NLP tasks with a focus on English due to English-centric pre-training and limited multilingual data. In this work, we focus on the problem of translation, and while some multilingual LLMs claim to support for hundreds of languages, models often fail to provide high-quality responses for mid- and low-resource languages, leading to imbalanced performance heavily skewed in favor of high-resource languages. We introduce **X-ALMA**, a model designed to ensure top-tier performance across 50 diverse languages, regardless of their resource levels. X-ALMA surpasses state-of-the-art open-source multilingual LLMs, such as Aya-101 and Aya-23, in every single translation direction on the FLORES-200 and WMT'23 test datasets according to COMET-22. This is achieved by plug-and-play language-specific module architecture to prevent language conflicts during training and a carefully designed training regimen with novel optimization methods to maximize the translation performance. After the final stage of training regimen, our proposed **A**daptive **R**ejection **P**reference **O**ptimization (**ARPO**) surpasses existing preference optimization methods in translation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。