用强化学习优化翻译模型，仅靠通用语料和专家反馈就能提升质量。
Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
- 基于偏好优化的强化学习框架，无需平行语料即可改进翻译。
- 在英德翻译上将Gemma3-1b的COMET得分从0.703提升至0.747。
- 适合想用少量人工或AI反馈优化现成翻译模型的研究者。
当前神经机器翻译系统几乎完全依赖监督平行数据训练，尽管取得显著进展，仍存在持续的翻译错误。本文提出一种基于强化学习的后训练范式，可有效修正此类错误。我们引入一个新框架，仅需通用文本语料库和一位专家（人类或AI）提供迭代反馈。实验聚焦于英语到德语这一典型高资源语言对。关键的是，我们采用直接偏好优化（DPO）实现该强化学习后训练。将该DPO驱动框架应用于Gemma3-1b模型，在英德翻译任务上其COMET得分从0.703提升至0.747。结果表明，DPO为通过偏好驱动的后训练提供了高效且稳定的路径，可有效提升预训练翻译模型性能。
原文摘要 · Abstract (English)
Contemporary neural machine translation (NMT) systems are almost exclusively built by training on supervised parallel data. Despite the tremendous progress achieved, these systems still exhibit persistent translation errors. This paper proposes that a post-training paradigm based on reinforcement learning (RL) can effectively rectify such mistakes. We introduce a novel framework that requires only a general text corpus and an expert translator which can be either human or an AI system to provide iterative feedback. In our experiments, we focus specifically on English-to-German translation as a representative high-resource language pair. Crucially, we implement this RL-based post-training using Direct Preference Optimization (DPO). Applying our DPO-driven framework to the gemma3-1b model yields a significant improvement in translation quality, elevating its COMET score from 0.703 to 0.747 on the English to German task. The results demonstrate that DPO offers an efficient and stable pathway for enhancing pre-trained NMT models through preference-based post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。