用动态提示词提升大模型多语言能力，效果显著
PolyPrompt: Automating Knowledge Extraction from Multilingual Language Models with Dynamic Prompt Generation
- 通过梯度搜索为每种语言生成专用触发词，推理时自动匹配
- 在15种语言上平均提升3.7%~19.9%准确率
- 无需参数修改，适合快速增强多语言模型性能
大语言模型在英语基准测试中表现优异，但在多语言场景下性能不一。为此，我们提出PolyPrompt，一种参数高效、可自动增强多语言能力的新框架。该方法通过梯度搜索为每种语言学习一组触发词，在推理时根据输入语种选择对应触发词并前置到提示词中。我们在两个约10亿参数的模型上进行实验，评估了全球MMLU基准在十五种类型和资源差异较大的语言上的表现，相比直接使用和翻译流水线基线，准确率提升3.7%至19.9%。
原文摘要 · Abstract (English)
Large language models (LLMs) showcase increasingly impressive English benchmark scores, however their performance profiles remain inconsistent across multilingual settings. To address this gap, we introduce PolyPrompt, a novel, parameter-efficient framework for enhancing the multilingual capabilities of LLMs. Our method learns a set of trigger tokens for each language through a gradient-based search, identifying the input query's language and selecting the corresponding trigger tokens which are prepended to the prompt during inference. We perform experiments on two ~1 billion parameter models, with evaluations on the global MMLU benchmark across fifteen typologically and resource diverse languages, demonstrating accuracy gains of 3.7%-19.9% compared to naive and translation-pipeline baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。