用最小贝叶斯风险解码提升无参考翻译评估的错误定位精度
Minimum Bayes Risk Decoding for Error Span Detection in Reference-Free Automatic Machine Translation Evaluation
- 采用最小贝叶斯风险解码，基于相似度选择更接近人工标注的候选答案
- 在WMT24任务中，跨度级性能显著提升，系统级和句子级表现优于或持平MAP
- 通过模型蒸馏将MBR决策转为贪婪搜索，降低推理延迟
错误跨度检测（ESD）通过定位翻译错误并标注其严重程度，扩展了自动机器翻译评估。现有生成式ESD方法通常使用最大后验概率（MAP）解码，假设模型估计的概率与人工标注相似度完全相关，但实际常出现错误标注的似然值高于人工标注。本文改用最小贝叶斯风险（MBR）解码，利用句级或跨度级相似度函数，根据候选假设与人工标注的近似相似度进行选择。在WMT24评测任务上的实验表明，MBR解码显著提升了跨度级性能，在系统级和句子级上通常达到或超过MAP表现。为降低MBR解码的计算开销，进一步将MBR决策蒸馏至可通过贪婪搜索解码的模型中，消除推理时延瓶颈。
原文摘要 · Abstract (English)
Error Span Detection (ESD) extends automatic machine translation (MT) evaluation by localizing translation errors and labeling their severity. Current generative ESD methods typically use Maximum a Posteriori (MAP) decoding, assuming that the model-estimated probabilities are perfectly correlated with similarity to the human annotation, but we often observe higher likelihood assigned to an incorrect annotation than to the human one. We instead apply Minimum Bayes Risk (MBR) decoding to generative ESD. We use a sentence- or span-level similarity function for MBR decoding, which selects candidate hypotheses based on their approximate similarity to the human annotation. Experimental results on the WMT24 Metrics Shared Task show that MBR decoding significantly improves span-level performance and generally matches or outperforms MAP at the system and sentence levels. To reduce the computational cost of MBR decoding, we further distill its decisions into a model decoded via greedy search, removing the inference-time latency bottleneck.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。