发现大模型在组合推理中存在线性关联，能解释幻觉成因。
Linear Correlation in LM's Compositional Generalization and Hallucination
- 通过线性变换建模知识组合，如城市到国家的映射
- 该线性关系在微调后仍保持，与真实世界一致时准确，偏离则产生幻觉
- 仅用前馈网络和预训练词表即可学习，说明模型泛化依赖词表
语言模型（LMs）的泛化能力引发广泛讨论，其在基本知识组合（如反向/过渡困境）方面表现不佳。本文揭示了语言模型在知识组合中存在线性相关性：特定相关知识间存在线性变换，可将一个提示的下一个词预测得分映射到另一个提示，例如对任意X，“X住的城市”→“X住的国家”。这与人类知识组合的线性特征一致，如巴黎→法国。实证表明，该线性变换对大规模微调具有鲁棒性，当与现实关系一致时可泛化新知识，但偏离时会引发幻觉。结果表明，线性相关性或可作为模型泛化的潜在指标。最后，我们证明这种线性相关性可通过单个前馈网络结合预训练词表表示学习，说明模型泛化高度依赖预训练词表。
原文摘要 · Abstract (English)
The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition (e.g., reverse/transition curse). This paper uncovers the phenomenon of linear correlations in LMs during knowledge composition. For explanation, there exists a linear transformation between certain related knowledge that maps the next token prediction logits from one prompt to another, e.g., "X lives in the city of" $\rightarrow$ "X lives in the country of" for every given X. This mirrors the linearity in human knowledge composition, such as Paris $\rightarrow$ France. Our findings indicate that the linear transformation is resilient to large-scale fine-tuning, generalizing updated knowledge when aligned with real-world relationships, but causing hallucinations when it deviates. Empirical results suggest that linear correlation can serve as a potential identifier of LM's generalization. Finally, we show such linear correlations can be learned with a single feedforward network and pre-trained vocabulary representations, indicating LM generalization heavily relies on the latter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。