发现大模型中概念与语言可分离，用激活修补实现跨语言概念迁移。
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
- 通过激活修补技术，在翻译任务中分离概念与语言表征。
- 同一概念在不同语言间平均表征可提升翻译效果。
- 模型能基于通用概念表征生成自然语言描述。
多语言建模的核心问题在于大型语言模型是否发展出脱离特定语言的通用概念表征。本文通过分析变压器架构大模型在词翻译任务中的潜在表示（latents），采用策略性地从源语言提示中提取潜在表示并插入目标语言提示的前向传播过程。结果显示，输出语言的编码早于待翻译概念的编码。基于此，我们开展两项关键实验：首先，仅通过激活修补即可独立改变概念或语言；其次，使用跨语言的同一概念平均表示进行修补，不仅不影响翻译能力，反而提升其表现。最后，我们将结果推广至多词生成任务，证明模型可基于这些平均表示生成自然语言描述。研究为所考察模型中存在语言无关的概念表征提供了证据。
原文摘要 · Abstract (English)
A central question in multilingual language modeling is whether large language models (LLMs) develop a universal concept representation, disentangled from specific languages. In this paper, we address this question by analyzing latent representations (latents) during a word-translation task in transformer-based LLMs. We strategically extract latents from a source translation prompt and insert them into the forward pass on a target translation prompt. By doing so, we find that the output language is encoded in the latent at an earlier layer than the concept to be translated. Building on this insight, we conduct two key experiments. First, we demonstrate that we can change the concept without changing the language and vice versa through activation patching alone. Second, we show that patching with the mean representation of a concept across different languages does not affect the models' ability to translate it, but instead improves it. Finally, we generalize to multi-token generation and demonstrate that the model can generate natural language description of those mean representations. Our results provide evidence for the existence of language-agnostic concept representations within the investigated models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。