arXiv:2502.14429cs.CL2025-02被引 3

提出轻量级翻译质量评估模型,实现快速判断与低计算开销。

Early-Exit and Instant Confidence Translation Quality Estimation

  • 基于早期退出机制,在模型浅层生成质量评分与置信度。
  • 计算成本降低50%,性能损失微小,适用于大规模评估与重排。
  • 可指导人工标注优先级,减少无效评估工作量。

质量评估在机器翻译中广泛应用,但现有模型通常不透明且计算开销大,难以用于大规模流水线。本文针对两个关联挑战:(1)降低大规模质量评估的计算成本;(2)构建廉价的不确定性估计方法。为此,提出 Instant Confidence COMET,一种具备置信度感知的质量评估模型,在远低于以往方法的计算成本下达到相当性能。进一步扩展为 Early-Exit COMET,可在模型早期层输出质量分数与置信度,支持提前终止计算,显著降低评估开销。该方法还应用于翻译重排,结合上置信度边界带算法,在无需对所有候选进行完整评估的情况下筛选最优译文。在评估与重排任务中,计算量均减少50%以上,性能下降极小。最后展示 Instant Confidence COMET 可用于识别应由人工标注的翻译,而非依赖单一 COMET 分数。

原文摘要 · Abstract (English)

Quality estimation is omnipresent in machine translation, for both evaluation and generation. Unfortunately, quality estimation models are often opaque and computationally expensive, making them impractical to be part of large-scale pipelines. In this work, we tackle two connected challenges: (1) reducing the cost of quality estimation at scale, and (2) developing an inexpensive uncertainty estimation method for quality estimation. To address the latter, we introduce Instant Confidence COMET, an uncertainty-aware quality estimation model that matches the performance of previous approaches at a fraction of their costs. We extend this to Early-Exit COMET, a quality estimation model that can compute quality scores and associated confidences already at early model layers, allowing us to early-exit computations and reduce evaluation costs. We also apply our model to machine translation reranking. We combine Early-Exit COMET with an upper confidence bound bandit algorithm to find the best candidate from a large pool without having to run the full evaluation model on all candidates. In both cases (evaluation and reranking) our methods reduce the required compute by 50% with very little degradation in performance. Finally, we show how Instant Confidence COMET can be used to decide which translations a human evaluator should score rather than relying on the COMET score.

质量评估早期退出不确定性估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。