arXiv:2608.03077cs.CL2026-08

让翻译模型学会分步决策,提升多领域翻译准确性。

PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation

论文配图:PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation
图 1 · 摘自论文原文
  • 用分步推理+强化学习,让每一步翻译都对最终结果负责。
  • 在15个领域、4种语言方向上平均提升翻译质量,尤其改善长文本和难句。
  • 适合需要精准术语与风格控制的翻译场景,如法律、医疗等专业领域。

多领域机器翻译不仅要求流畅生成,还需具备领域敏感性,如领域消歧、术语控制和风格适配。大推理模型(LRMs)通过中间翻译步骤显式表达决策,但我们在15个领域和四种翻译方向上的分析发现:这种显式推理虽提升了长文本和高难度翻译的表现,但在术语密集和风格受限场景下常出现偏差。根源在于信用分配瓶颈——现有方法优化最终输出或粗粒度轨迹,却无法识别哪些中间步骤真正有助于最终翻译。为此,我们提出PAMT,一种过程对齐训练框架,结合冷启动领域感知的长思维链监督与强化学习。PAMT使用序列级格式和结果奖励,同时引入步骤级过程奖励,衡量每一步翻译对参考译文可能性的提升程度。在两种骨干模型上,PAMT均优于基线模型,在平均表现上超越专用翻译模型,并在域内、域外及多语言设置下保持与强大多模型/推理模型相当的竞争力。

原文摘要 · Abstract (English)

Multi-domain machine translation (MDMT) requires more than fluent generation: it demands domain-sensitive translation decisions such as domain disambiguation, terminology control, and stylistic adaptation. Large reasoning models (LRMs) make such decisions explicit through intermediate translation steps, but our analysis across 15 domains and four translation directions shows that this explicit reasoning is double-edged: it improves long-form and high-difficulty translation, yet often drifts in terminology-intensive and stylistically constrained settings. We trace this failure to a credit-assignment bottleneck: existing methods optimize final outputs or coarse trajectories, but cannot identify which translation steps actually help the final translation. To address this, we propose PAMT, a process-aligned training framework that combines cold-start domain-aware Long-CoT supervision with reinforcement learning. PAMT uses sequence-level format and outcome rewards for the final translation, together with a step-level process reward that measures how much each explicit translation step increases the likelihood of the reference translation. Across two backbones, PAMT improves over base models, outperforms MT-specialized baselines on average, and remains competitive with strong LLMs/LRMs across in-domain, OOD, and multilingual settings.

多领域翻译强化学习推理模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。