长文本翻译评估中，输入长度影响LLM判断，新方法可缓解偏差。
Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models
- 用长文本输入LLM评估翻译质量，发现越长越少报错。
- 改进提示和微调后，评估结果更稳定，系统排名准确率提升。
- 适合需要长文档翻译质检的研究者与工业用户。
准确评估机器翻译质量仍是长期挑战，尤其在长文档场景下。近期研究显示，大语言模型（LLMs）可通过MQM错误段标注，可靠地进行句子级翻译评估。随着现代LLMs支持更大的上下文窗口，一个自然问题是：能否将整个文档的译文输入LLM进行质量评估？理想情况下，评估结果应与文本长度无关，无论输入是短句还是整篇文档，都应产出一致的错误段。然而我们的分析表明，文本长度显著影响评估结果：更长的文本导致错误段数量减少，系统排名准确性下降。为解决这一局限，我们评估了多种策略，包括粒度对齐提示、焦点句提示（FSP）以及一种微调方法，以更好地使LLM适配评估任务。后两种方法有效缓解了长度偏差，使LLM在长文本翻译评估中更具可靠性。
原文摘要 · Abstract (English)
Accurately evaluating machine-translated text remains a long-standing challenge, particularly for long documents. Recent work has shown that large language models (LLMs) can serve as reliable and interpretable sentence-level translation evaluators via MQM error span annotations. With modern LLMs supporting larger context windows, a natural question arises: can we feed entire document translations into an LLM for quality assessment? Ideally, evaluation should be invariant to text length, producing consistent error spans regardless of input granularity. However, our analysis shows that text length significantly impacts evaluation: longer texts lead to fewer error spans and reduced system ranking accuracy. To address this limitation, we evaluate several strategies, including granularity-aligned prompting, Focus Sentence Prompting (FSP), and a fine-tuning approach to better align LLMs with the evaluation task. The latter two methods largely mitigate this length bias, making LLMs more reliable for long-form translation evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。