用简单偏移让大模型切换语言，无需更新参数即可提升多语上下文学习效果。
Language Steering for Multilingual In-Context Learning
- 通过计算中英对照例句的激活差异，生成语言向量作为推理时的偏移量。
- 在19种语言、3个模型上均实现稳定提升，最高性能超越基线12.7%。
- 向量可解释性强，同类语言聚集紧密，跨任务迁移有效，适合多语应用研究者。
若大语言模型处于统一语义空间，则语言切换应仅需简单激活偏移。为此，我们以多语上下文学习为案例，即训练示例用英语但测试查询为目标语言。提出语言向量：在特定层计算平行源语言与目标语言示例的平均激活差异，推理时作为隐藏状态偏移量，引导模型内部表征向目标语言对齐。在覆盖19种语言、3个模型的三项多语任务上评估，方法在所有任务和语言中均持续优于基线，证明无需参数更新，仅靠激活偏移即可有效重定向模型语言模式。此外，向量编码可解释的语言结构：相近语言形成紧密聚类，且向量可在不同任务间迁移，表明语言身份在模型激活空间中占据可分离且结构化的方向。
原文摘要 · Abstract (English)
If large language models operate in a universal semantic space, then switching between languages should require only a simple activation offset. To test this, we take multilingual in-context learning as a case study, where few-shot demonstrations are provided in English but the test query is in a target language. We propose language vectors, computed as the mean activation difference between parallel source and target language examples at a particular layer, and added as an offset to hidden states at inference time to shift the model's internal representations toward the target language. We evaluate our method across three multilingual tasks spanning 19 languages and three models. Our results show consistent improvements on multilingual in-context learning over baselines across all tasks and languages tested, demonstrating that a simple activation offset is sufficient to redirect a model's language mode without any parameter updates. Beyond performance, the vectors encode interpretable linguistic structure, with closely related languages forming tight clusters and vectors transferring across tasks, suggesting that language identity occupies separable and structured directions in a model's activation space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。