研究多语言大模型中提示翻译的最佳策略,发现不同语言和任务下翻译方式差异显著。
Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs
- 将提示拆分为指令、上下文、示例、输出四部分,逐项测试翻译效果
- 在35种语言上验证,发现翻译质量与语系相似性影响模型表现
- 为低资源语言提供针对性翻译建议,适合多语言应用开发者参考
尽管大型语言模型在多种任务上的多语言能力不断进步,英语仍是研究与开发的主导语言。因此,实践中普遍采用预翻译策略——即在推理前将任务提示翻译成英语。选择性预翻译则更精细,仅翻译特定提示组件,但当前使用零散,缺乏系统研究基础。本文系统评估预翻译策略,将提示视为由指令、上下文、示例和输出组成的模块化结构,分别尝试是否翻译各部分。我们在涵盖高低资源语言的35种语言上,对问答(QA)、自然语言推断(NLI)、命名实体识别(NER)和摘要生成等任务进行实验。结果表明,与英语的语系相似性、翻译质量及预训练数据规模是影响模型性能的关键因素。本文提出适用于不同多语言场景的实用指导策略。
原文摘要 · Abstract (English)
Despite advances in the multilingual capabilities of Large Language Models (LLMs) across diverse tasks, English remains the dominant language for LLM research and development. So, when working with a different language, this has led to the widespread practice of pre-translation, i.e., translating the task prompt into English before inference. Selective pre-translation, a more surgical approach, focuses on translating specific prompt components. However, its current use is sporagic and lacks a systematic research foundation. Consequently, the optimal pre-translation strategy for various multilingual settings and tasks remains unclear. In this work, we aim to uncover the optimal setup for pre-translation by systematically assessing its use. Specifically, we view the prompt as a modular entity, composed of four functional parts: instruction, context, examples, and output, either of which could be translated or not. We evaluate pre-translation strategies across 35 languages covering both low and high-resource languages, on various tasks including Question Answering (QA), Natural Language Inference (NLI), Named Entity Recognition (NER), and Abstractive Summarization. Our experiments show the impact of factors as similarity to English, translation quality and the size of pre-trained data, on the model performance with pre-translation. We suggest practical guidelines for choosing optimal strategies in various multilingual settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。