无需平行数据，用强化学习提升机器翻译性能，尤其适合低资源语言。
Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective
- 采用无参考奖励机制，在无并行数据下优化编码器-解码器模型。
- 在13种语言上均提升，中文最高增5.03点chrF++，媲美有监督微调。
- 对低资源、复杂形态语言效果最佳，适合数据稀缺场景。
生产级机器翻译主要依赖编码器-解码器序列到序列（Seq2Seq）模型，但现有强化学习微调方法多针对参数量≥70亿的解码器仅模型，对编码器-解码器架构研究有限。本文将分组相对策略优化（GRPO）应用于NLLB-200（600M和1.3B参数），使用结合LaBSE与COMET-Kiwi的混合无参考奖励，微调时无需并行数据，覆盖13种语系多样语言。实验显示，GRPO在全部13种语言上均带来一致提升，传统中文最高达+5.03 chrF++；在不使用目标语言数据的情况下，其表现可媲美3轮有监督微调，尤其在形态复杂的语言中效果显著。我们发现，性能提升最大的情况出现在基线表现最弱、奖励区分度最高的场景，表明该方法在平行数据最稀缺时最具优势，并在英译和西译源语言上重复验证了这一规律。
原文摘要 · Abstract (English)
Production machine translation relies overwhelmingly on encoder-decoder Seq2Seq models, yet reinforcement learning approaches to MT fine-tuning have largely targeted decoder-only LLMs at $\geq$7B parameters, with limited systematic study of encoder-decoder architectures. We apply Group Relative Policy Optimization to NLLB-200 (600M and 1.3B) using a hybrid reference-free reward (LaBSE and COMET-Kiwi) that requires no parallel data at fine-tuning time, evaluating across 13 typologically diverse languages. GRPO yields consistent improvements on all 13 languages, up to $+$5.03 chrF++ for Traditional Chinese, and, without any target-language data, competes with 3-epoch supervised fine-tuning on morphologically complex languages . We identify a consistent empirical pattern in which gains are largest where baseline performance is weakest and reward discriminability is highest, making this approach most effective precisely where parallel data is scarcest, and replicate this pattern across English and Spanish source languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。