用强化学习训练模型重写源文本，提升机器翻译质量。
Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation
- 用下游翻译质量作为奖励信号，训练重写模型
- 40亿参数模型在16个语言对上超越小模型提示重写
- 无需大模型即可达到2350亿参数模型的水平
先前研究探索了使用大语言模型（LLMs）对源文本进行重写以提升机器翻译（MT）质量。然而我们发现，这种基于提示的重写方法在使用较小的LLM（如40亿参数模型）时反而会降低翻译质量。问题根源在于仅通过自然语言提示难以控制重写行为：重写只有在带来更好翻译结果时才有价值，但现有方法未显式优化这一目标。为此，我们提出RLSR（强化学习用于源文本重写），一个以下游翻译质量提升为奖励信号的强化学习框架。在六个机器翻译模型和16个语言对上的实验表明，我们的40亿参数RLSR训练的重写模型显著优于不重写的基线和同规模提示重写基线，同时与使用2350亿参数大模型的基线表现相当。代码与模型已公开于https://github.com/vlaks425/MT-RLSR。
原文摘要 · Abstract (English)
Prior work has explored prompting large language models (LLMs) to rewrite source text before translation, with the goal of improving machine translation (MT) quality. However, we find that such prompt-based rewriting can degrade translation quality rather than enhance it, particularly when smaller LLMs, such as 4B-parameter models, are used. We argue that this limitation stems from the difficulty of controlling rewriting behavior through natural-language prompts alone: a rewrite is useful only if it leads to a better downstream translation, yet existing prompt-based methods do not explicitly optimize for this signal. To address this issue, we propose \textbf{RLSR} (\textbf{R}einforcement \textbf{L}earning for \textbf{S}ource \textbf{R}ewriting), a reinforcement learning framework that trains the rewriting model with a reward derived from the downstream translation-quality improvement produced by each rewrite. Experiments across six MT models and 16 language pairs show that our 4B RLSR-trained rewriting models significantly outperform both the no-rewriting baseline and same-scale prompt-based rewriting baselines, while remaining competitive with baselines that use a 235B LLM. Our models and code are available at: https://github.com/vlaks425/MT-RLSR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。