用生成参考文本+语义相似度评估翻译质量，效果优于直接打分。
LLMs Are Not Scorers: Rethinking MT Evaluation with Generation-Based Methods
- 先让大模型生成高质量参考译文，再用嵌入向量比对语义相似度。
- 在8种语言对和8个大模型上测试，相关性显著高于直接评分法。
- 适合需要高精度自动评估的翻译研究与工业落地场景。
近期研究尝试通过提示大语言模型（LLM）直接给出翻译质量分数，但这类方法在句子级别与人工判断的相关性较低。本文提出一种基于生成的评估范式：利用解码器仅有的大语言模型生成高质量参考译文，随后使用句子嵌入进行语义相似度评分。我们在机器翻译质量评估领域进行了迄今为止最广泛的实验，涵盖8个大语言模型和8种语言对。实证结果表明，该方法在段落级相关性上优于同模型直接评分基线，也优于来自MTME的非大模型参考无监督指标。这些发现凸显了生成式评估的优势，支持向结合流畅生成与精准语义评估的混合方法转变。
原文摘要 · Abstract (English)
Recent studies have applied large language models (LLMs) to machine translation quality estimation (MTQE) by prompting models to assign numeric scores. Nonetheless, these direct scoring methods tend to show low segment-level correlation with human judgments. In this paper, we propose a generation-based evaluation paradigm that leverages decoder-only LLMs to produce high-quality references, followed by semantic similarity scoring using sentence embeddings. We conduct the most extensive evaluation to date in MTQE, covering 8 LLMs and 8 language pairs. Empirical results show that our method outperforms both intra-LLM direct scoring baselines and external non-LLM reference-free metrics from MTME. These findings demonstrate the strength of generation-based evaluation and support a shift toward hybrid approaches that combine fluent generation with accurate semantic assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。