放大特定语言神经元可有效引导大模型输出目标语言，尤其对低资源语言有益。
Unveiling the Influence of Amplifying Language-Specific Neurons
- 通过干预放大18种语言的特异性神经元，提升语言导向能力
- 最优放大因子使模型在多数语言上输出更准确，但跨语言表现下降
- 对低资源语言有显著帮助，适合专注单一语言场景的应用
大型语言模型中与特定语言强相关的语言特异性神经元已被证实可通过关闭影响模型行为，但其放大效应尚不明确。本文在18种语言（含低资源语言）上，针对三种主要训练于不同语言的模型，通过干预放大这些神经元，并采用新提出的语言引导转移（LSS）评分评估其有效性。结果表明，最优放大因子能有效引导输出至几乎所有测试语言。在下游任务中，该干预提升了部分语言的自语言性能，但总体削弱了跨语言表现。研究揭示了语言特异性神经元在多语言行为中的关键作用，放大策略对低资源语言尤为有利，但在跨语言迁移中优势有限。
原文摘要 · Abstract (English)
Language-specific neurons in LLMs that strongly correlate with individual languages have been shown to influence model behavior by deactivating them. However, their role in amplification remains underexplored. This work investigates the effect of amplifying language-specific neurons through interventions across 18 languages, including low-resource ones, using three models primarily trained in different languages. We compare amplification factors by their effectiveness in steering to the target language using a proposed Language Steering Shift (LSS) evaluation score, then evaluate it on downstream tasks: commonsense reasoning (XCOPA, XWinograd), knowledge (Include), and translation (FLORES). The optimal amplification factors effectively steer output toward nearly all tested languages. Intervention using this factor on downstream tasks improves self-language performance in some cases but generally degrades cross-language results. These findings highlight the effect of language-specific neurons in multilingual behavior, where amplification can be beneficial especially for low-resource languages, but provides limited advantage for cross-lingual transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。