arXiv:2602.08600cs.CL2026-02被引 1

用强化学习让翻译质量评估模型读懂错误类型,提升低资源语言表现

Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translation

  • 基于人类标注的错误描述和评分,构建首个英-马拉雅拉姆语段级质量评估数据集
  • 提出ALOPE-RL框架,在小规模数据上实现顶尖性能,仅需≤40亿参数模型
  • 通过错误感知奖励机制,让大模型不仅给分,还能理解具体翻译问题

质量评估(QE)旨在无需参考译文的情况下评估机器翻译质量,对真实场景的大规模评估至关重要。大语言模型(LLMs)在该领域展现出巨大潜力,但现有方法多依赖单一数值评分,无法揭示导致评分的具体翻译错误。针对低资源语言中标注数据稀缺的问题,本文首次构建了英-马拉雅拉姆语段级QE数据集,包含人工标注的直接评估(DA)分数与翻译质量评述(TQR),即描述翻译错误的上下文自由文本评论。提出ALOPE-RL框架,基于策略的强化学习方法,利用DA分数与TQR生成奖励信号,训练高效适配器。结合错误感知奖励,使LLM能超越数值评分进行推理。尽管训练数据量小,该方法在英-马拉雅拉姆语翻译质量评估中表现优异,使用不超过40亿参数的紧凑模型(经LoRA微调与4比特量化),优于更大规模的基线模型及主流编码器型QE模型。结果表明,错误感知的策略学习可在有限数据与算力条件下实现强大性能。数据集、代码与训练模型已公开。

原文摘要 · Abstract (English)

Quality Estimation (QE) aims to assess the quality of machine translation (MT) outputs without relying on reference translations, making it essential for real-world, large-scale MT evaluation. Large Language Models (LLMs) have shown significant promise in advancing the field of quality estimation of machine translation. However, most of the QE approaches solely rely on scalar quality scores, offering no explicit information about the translation errors that should drive these judgments. Moreover, for low-resource languages where annotated QE data is limited, existing approaches struggle to achieve reliable performance. To address these challenges, we introduce the first segment-level QE dataset for English to Malayalam, a severely resource-scarce language pair in the QE domain, comprising human-annotated Direct Assessment (DA) scores and Translation Quality Remarks (TQR), which are short, contextual, free-form annotator comments that describe translation errors. We further introduce ALOPE-RL, a policy-based reinforcement learning framework that trains efficient adapters based on policy rewards derived from DA score and TQR. Integrating error-aware rewards with ALOPE-RL, enables LLMs to reason about translation quality beyond numeric scores. Despite being trained on a small-scale QE dataset, ALOPE-RL achieves state-of-the-art performance on English to Malayalam QE using compact LLMs (<=4B parameters}) fine-tuned with LoRA and 4-bit quantization, outperforming both larger LLM-based baselines and leading encoder-based QE models. Our results demonstrate that error-aware, policy-based learning can deliver strong QE performance under limited data and compute budgets. We release our dataset, code, and trained models to support future research.

质量评估强化学习低资源语言错误分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。