发现大模型用线性子空间进行数字推理，可精准操控结果
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
- 通过偏最小二乘回归定位数字属性的线性子空间
- 干预子空间可改变模型对数字比较的判断结果
- 在3个模型上验证，适合研究模型内部机制的读者
本文研究大型语言模型（LLMs）在回答涉及数值比较的问题时，是否利用嵌入空间中低维子空间编码的数值属性。通过偏最小二乘回归方法，我们识别出与比较提示中实体相关的数值属性所编码的子空间。进一步实验表明，通过干预这些子空间以操纵隐藏状态，可导致模型的比较结果发生改变，证明了因果关系的存在。在三个不同大型语言模型上的实验均显示，该现象适用于多种数值属性，说明大语言模型确实利用线性编码的信息进行数值推理。
原文摘要 · Abstract (English)
This paper investigates whether large language models (LLMs) utilize numerical attributes encoded in a low-dimensional subspace of the embedding space when answering questions involving numeric comparisons, e.g., Was Cristiano born before Messi? We first identified, using partial least squares regression, these subspaces, which effectively encode the numerical attributes associated with the entities in comparison prompts. Further, we demonstrate causality, by intervening in these subspaces to manipulate hidden states, thereby altering the LLM's comparison outcomes. Experiments conducted on three different LLMs showed that our results hold across different numerical attributes, indicating that LLMs utilize the linearly encoded information for numerical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。