用强化学习训练大模型实现自由翻译，无需人工标注。
DeepTrans: Deep Reasoning Translation via Reinforcement Learning
- 通过预设评分标准构建奖励模型，指导模型思考与自由翻译
- 基于Qwen2.5-7B，文学翻译性能提升16.3%，超越强基线模型
- 无需标签数据，适合对翻译质量要求高的研究者
近期，深度推理大模型（如OpenAI o1和DeepSeek-R1）在多项下游任务中表现优异。自由翻译是多语言世界中的重要且有趣任务，需突破逐字翻译的局限，但该任务在深度推理大模型中仍鲜有探索。本文提出DeepTrans，一种通过强化学习（RL）学习自由翻译的深度推理翻译模型。我们精心构建了一个奖励模型，基于翻译结果与思维过程的预设评分标准进行评估。该奖励模型在强化学习过程中指导DeepTrans如何思考并实现自由翻译。此外，我们的训练无需任何标注翻译数据，避免了人工标注或高成本数据合成。实验表明，以Qwen2.5-7B为骨干网络，DeepTrans在文学翻译任务上性能提升16.3%，优于多个强基线深度推理模型。同时，我们总结了强化学习探索中的失败案例与有趣发现，希望激励更多研究者关注自由翻译任务。
原文摘要 · Abstract (English)
Recently, deep reasoning LLMs (e.g., OpenAI o1 and DeepSeek-R1) have shown promising performance in various downstream tasks. Free translation is an important and interesting task in the multilingual world, which requires going beyond word-for-word translation. However, the task is still under-explored in deep reasoning LLMs. In this paper, we introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning (RL). Specifically, we carefully build a reward model with pre-defined scoring criteria on both the translation results and the thought processes. The reward model teaches DeepTrans how to think and free-translate the given sentences during RL. Besides, our RL training does not need any labeled translations, avoiding the human-intensive annotation or resource-intensive data synthesis. Experimental results show the effectiveness of DeepTrans. Using Qwen2.5-7B as the backbone, DeepTrans improves performance by 16.3% in literature translation, and outperforms strong deep reasoning LLMs. Moreover, we summarize the failures and interesting findings during our RL exploration. We hope this work could inspire other researchers in free translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。