arXiv:2602.04764cs.CL2026-02被引 2

用上万条示例测试大模型翻译低资源语言,发现更多例子不等于更好效果。

Beyond Many-Shot Translation: Scaling In-Context Demonstrations For Low-Resource Machine Translation

  • 用超长上下文让大模型看数千个翻译示例来提升低资源语言翻译
  • 增加示例数量在特定条件下会快速饱和甚至导致性能下降
  • 单语数据有时比双语数据还有效,适合关注少样本翻译的开发者

为低资源语言构建机器翻译系统面临高质量数据稀缺的挑战。尽管大语言模型(LLMs)提升了翻译性能,但将其适配到少数语言仍困难。上下文学习(ICL)通过推理时提供示范,或可解决此问题。本研究将低资源机器翻译的ICL扩展至数千示例,使用长上下文模型将上下文令牌预算提升至100万。对比三种训练语料作为上下文监督:单语无监督数据、指令式数据和双语平行数据(英语-目标语、印尼语-目标语)。在爪哇语和巽他语上的实验表明,额外上下文带来的增益迅速饱和,接近最大上下文窗口时性能可能下降,且表现高度依赖语料类型。值得注意的是,某些单语监督形式在效果上可媲美双语数据,尽管后者提供了额外监督。总体而言,研究揭示了长上下文ICL在低资源翻译中的有效边界与语料敏感性,说明更大的上下文窗口未必带来成比例的质量提升。

原文摘要 · Abstract (English)

Building machine translation (MT) systems for low-resource languages is notably difficult due to the scarcity of high-quality data. Although Large Language Models (LLMs) have improved MT system performance, adapting them to lesser-represented languages remains challenging. In-context learning (ICL) may offer novel ways to adapt LLMs for low-resource MT by conditioning models on demonstration at inference time. In this study, we explore scaling low-resource machine translation ICL beyond the few-shot setting to thousands of examples with long-context models. We scale in-context token budget to 1M tokens and compare three types of training corpora used as in-context supervision: monolingual unsupervised data, instruction-style data, and parallel data (English--target and Indonesian--target). Our experiments on Javanese and Sundanese show that gains from additional context saturate quickly and can degrade near the maximum context window, with scaling behavior strongly dependent on corpus type. Notably, some forms of monolingual supervision can be competitive with parallel data, despite the latter offering additional supervision. Overall, our results characterize the effective limits and corpus-type sensitivity of long-context ICL for low-resource MT, highlighting that larger context windows do not necessarily yield proportional quality gains.

低资源翻译上下文学习大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。