发现不同语言下模型用相同机制处理主谓一致，关键信号跨语言通用。
On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task
- 通过分析注意力头与MLP的协同，定位出主谓一致的核心信号路径。
- 找到一个在英西语中都存在的残差空间方向，能直接控制预测结果。
- 该机制在同系列多个模型中复现，说明具有通用性，适合研究模型可解释性。
近期已有若干语言模型的算法被成功逆向工程，但主要集中于特定任务和模型，其电路的普遍性尚不明确。本文研究了Gemma 2B在英语和西班牙语中解决主谓一致任务时的内部电路。发现两个语言下的电路高度一致：由特定注意力头将“主语数”信号写入最后残差流，再由最终MLP中的少量神经元读取。该信号以残差空间中的一个方向表示，具有语言无关性。我们验证了该方向对模型预测具有因果效应：通过干预英语中发现的方向，可直接翻转西班牙语的动词数预测。此外，在Gemma 1和Gemma 2系列的其他模型中也发现了类似行为。
原文摘要 · Abstract (English)
Several algorithms implemented by language models have recently been successfully reversed-engineered. However, these findings have been concentrated on specific tasks and models, leaving it unclear how universal circuits are across different settings. In this paper, we study the circuits implemented by Gemma 2B for solving the subject-verb agreement task across two different languages, English and Spanish. We discover that both circuits are highly consistent, being mainly driven by a particular attention head writing a `subject number' signal to the last residual stream, which is read by a small set of neurons in the final MLPs. Notably, this subject number signal is represented as a direction in the residual stream space, and is language-independent. We demonstrate that this direction has a causal effect on the model predictions, effectively flipping the Spanish predicted verb number by intervening with the direction found in English. Finally, we present evidence of similar behavior in other models within the Gemma 1 and Gemma 2 families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。