用批量提示和压缩技术,让机器翻译评估更省 tokens 且保持精度。
BatchGEMBA: Token-Efficient Machine Translation Evaluation with Batched Prompting and Prompt Compression
- 将多个翻译样本合并成一个提示,减少 2-4 倍 token 消耗。
- 提出批处理感知的提示压缩模型,平均再降 13-15% token。
- 在大批次下仍能稳住性能,适合高效评估场景。
基于大语言模型的自然语言生成评估近年多采用单例提示,导致大量 token 开销与计算低效。本文提出 BatchGEMBA-MQM 框架,结合批量提示与 GEMBA-MQM 评估指标。通过将多个翻译样本聚合至单一提示,相较单例提示可降低 2-4 倍 token 使用量(取决于批次大小)。此外,我们设计了批处理感知的提示压缩模型,在平均层面额外减少 13-15% token,同时缓解批处理引发的质量下降。在 GPT-4o、GPT-4o-mini、Mistral Small、Phi4 与 CommandR7B 等多个 LLM 及不同批次规模下评估表明:尽管批量处理通常降低质量(但部分情况影响不大),提示压缩不会进一步损害性能,甚至在某些情况下恢复损失。例如,启用压缩时,GPT-4o 在批次为 4 的情况下仍保持超过 90% 的基线性能,而未压缩则下降至 44.6%。代码与训练模型将开源至 https://github.com/NL2G/batchgemba,以支持该领域后续研究。
原文摘要 · Abstract (English)
Recent advancements in Large Language Model (LLM)-based Natural Language Generation evaluation have largely focused on single-example prompting, resulting in significant token overhead and computational inefficiencies. In this work, we introduce BatchGEMBA-MQM, a framework that integrates batched prompting with the GEMBA-MQM metric for machine translation evaluation. Our approach aggregates multiple translation examples into a single prompt, reducing token usage by 2-4 times (depending on the batch size) relative to single-example prompting. Furthermore, we propose a batching-aware prompt compression model that achieves an additional token reduction of 13-15% on average while also showing ability to help mitigate batching-induced quality degradation. Evaluations across several LLMs (GPT-4o, GPT-4o-mini, Mistral Small, Phi4, and CommandR7B) and varying batch sizes reveal that while batching generally negatively affects quality (but sometimes not substantially), prompt compression does not degrade further, and in some cases, recovers quality loss. For instance, GPT-4o retains over 90% of its baseline performance at a batch size of 4 when compression is applied, compared to a 44.6% drop without compression. We plan to release our code and trained models at https://github.com/NL2G/batchgemba to support future research in this domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。