arXiv:2411.02791cs.CLcs.AI2024-11被引 2

用大模型自反翻译,无需真实答案就能评估翻译质量。

Language Models and Cycle Consistency for Self-Reflective Machine Translation

  • 通过双向翻译检测一致性,间接评估目标语言译文质量。
  • 模型越大或推理次数越多,一致性越强,符合规模定律。
  • 适合评估大模型多语言翻译能力,无需双语数据。

本文提出一种基于大语言模型(LLM)的机器翻译新框架。核心思想是:理想的翻译应让足够强大的LLM能完整还原原文。我们从源语言A生成多个目标语言B的翻译候选,再将这些候选回译回A,通过词元级精度和准确率等指标评估原句与回译句的一致性,从而隐式衡量目标语言译文质量,无需真实参考答案。同时,仅需单语语料即可评估LLM的多语言翻译能力。对每个源句,选择与原文一致性最高的候选作为最终输出。实验表明,更大规模的LLM或更多前向推理次数均能提升一致性,符合模型规模扩展规律和测试时计算扩展规律。该方法实现了:1)在无真值情况下评估目标语言译文质量;2)仅用单语数据评估LLM的任意语言间翻译能力;3)为特定LLM生成更优翻译。

原文摘要 · Abstract (English)

This paper introduces a novel framework that leverages large language models (LLMs) for machine translation (MT). We start with one conjecture: an ideal translation should contain complete and accurate information for a strong enough LLM to recover the original sentence. We generate multiple translation candidates from a source language A to a target language B, and subsequently translate these candidates back to the original language A. By evaluating the cycle consistency between the original and back-translated sentences using metrics such as token-level precision and accuracy, we implicitly estimate the translation quality in language B, without knowing its ground-truth. This also helps to evaluate the LLM translation capability, only with monolingual corpora. For each source sentence, we identify the translation candidate with optimal cycle consistency with the original sentence as the final answer. Our experiments demonstrate that larger LLMs, or the same LLM with more forward passes during inference, exhibit increased cycle consistency, aligning with the LLM model size scaling law and test-time computation scaling law. This work provide methods for, 1) to implicitly evaluate translation quality of a sentence in the target language, 2), to evaluate capability of LLM for any-to-any-language translation, and 3), how to generate a better translation for a specific LLM.

机器翻译大模型自反射一致性评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。