用群体对比提升机器翻译评分精度，让模型更懂语言细微差别。
GRRM: Group Relative Reward Modeling for Machine Translation
- 通过联合分析候选译文群体，实现相对质量评估
- 在多个数据集上排名准确率超越基线方法
- 适合需要精细语义理解的翻译任务与大模型训练
尽管群体相对策略优化（GRPO）为大模型后训练提供了强大框架，其在开放域如机器翻译中的效果依赖于组内排序的准确性。我们发现，传统点对点质量度量（PQM）在此场景下表现不足：候选译文被孤立评估，缺乏比较上下文，难以区分细微语言差异。为此，我们提出群体质量度量（GQM）范式及其具体实现——群体相对奖励模型（GRRM）。不同于独立评分器，GRRM联合处理整个候选组，通过对比分析严格判断相对质量并自适应调整粒度。实验表明，GRRM在所有基线中达到具有竞争力的排名准确率；将其融入GRPO训练不仅提升了整体翻译质量，还解锁了与顶尖推理模型相当的推理能力。代码、数据集及模型权重已开源至 https://github.com/NJUNLP/GRRM。
原文摘要 · Abstract (English)
While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group ranking. We identify that standard Pointwise Quality Metrics (PQM) fall short in this context: candidates are evaluated in isolation, so the comparative context is missing for distinguishing fine-grained linguistic nuances. To address this, we introduce the Group Quality Metric (GQM) paradigm and its instantiation, the Group Relative Reward Model (GRRM). Unlike traditional independent scorers, GRRM jointly processes the entire candidate group, leveraging comparative analysis to rigorously resolve relative quality and adaptive granularity. Empirical evaluations confirm that GRRM achieves competitive ranking accuracy among all baselines; integrating GRRM into the GRPO training not only improves general translation quality but also unlocks reasoning capabilities comparable to state-of-the-art reasoning models. We release codes, datasets, and model checkpoints at https://github.com/NJUNLP/GRRM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。