arXiv:2511.13180cs.CL2025-11被引 1

提出翻译熵新指标,量化机器翻译系统的不确定性。

Translation Entropy: A Statistical Framework for Evaluating Translation Systems

  • 通过替换句子中单个词并观察译文不变性,统计其替换概率
  • 发现翻译熵随解码器层级提升,且双词替换呈乘积效应
  • 可用于客观评估翻译系统性能,适合研究者与工程师参考

自公元前3世纪以来,书面语言翻译已有悠久历史,但在信息时代其需求愈发普遍。尽管现有基于编码器-解码器深度架构的翻译系统众多,却缺乏定量评估方法,原因可能在于单一语言的熵值尚不明确。本文提出一种量化翻译熵的方法:对特定源句中的某一词进行替换,若译文保持不变,则记录该词可被替代的概率。通过对多个此类句子的统计分析,得到该词的翻译熵;所有词的平均熵即为系统整体翻译熵,结果显示其在解码器各层递增。该度量可用于对MarianMT、T5-Base和NLLB-200等公开翻译系统进行定量排序,并揭示互译熵是否对称。扩展至双词替换时,发现翻译退化程度与两词退化程度的乘积成正比。这些结果确立了翻译熵作为可测量属性和客观基准的可能性。

原文摘要 · Abstract (English)

The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures, nevertheless, no quantitative objective methods are available to assess their performance, likely because the entropy of even a single language remains unknown. This study presents a quantitative method for estimating translation entropy, with the following key finding. Given a translator, several sentences that differ by only one selected token of a given pivot sentence yield identical translations. Analyzing the statistics of this phenomenon across an ensemble of such sentences, consisting each of a pivot selected token, yields the probabilities of replacing this specific token with others while preserving the translation. These probabilities constitute the entropy of the selected token, and the average across all selected pivot tokens provides an estimate of the translator's overall translation entropy, which is enhanced along the decoder blocks. This entropic measure allows for the quantitative ranking of several publicly available translators and reveals whether mutual translation entropy is symmetric. Extending the proposed method to include the replacement of two tokens in a given pivot sentence demonstrates a multiplicative effect, where translation degeneracy is proportional to the product of the degeneracies of the two tokens. These findings establish translation entropy as a measurable property and objective benchmarking of artificial translators. Results are based on MarianMT, T5-Base and NLLB-200 translators.

机器翻译熵度量评估方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。