发现语言模型用回溯机制追踪角色信念，理解其心智理论能力。
Language Models use Lookbacks to Track Beliefs
- 通过构建因果故事数据集，发现模型用回溯机制记忆角色-对象-状态关系。
- 在角色不可见时，模型能正确更新信念，依赖可见性标识符进行信息检索。
- 揭示了模型内部如何动态绑定和召回信息，适合研究大模型认知机制者参考。
语言模型如何表征角色的信念，尤其是当这些信念与现实不一致时？这一问题关乎理解语言模型的元心理（ToM）能力。我们利用因果中介与抽象分析方法，探究模型推理角色信念的能力。构建了名为CausalToM的数据集，包含两个角色独立改变两个物体状态的简单故事，彼此可能未知对方行动。研究发现一种普遍存在的算法模式，称为回溯机制，使模型在需要时可召回关键信息。模型将每个角色-对象-状态三元组通过低秩子空间中的序号（OIs）关联起来。当被问及某角色对物体状态的信念时，绑定回溯会检索正确的状态OIs，再通过答案回溯获取对应状态标记。引入文本说明一角色是否可见另一角色后，模型先生成表示观察者与被观察者关系的可见性标识符。在可见性回溯中，该标识符用于检索被观察者的信息，并更新观察者信念。本工作为理解信念追踪机制提供了洞见，推动了对语言模型中元心理推理的逆向工程。
原文摘要 · Abstract (English)
How do language models (LMs) represent characters' beliefs, especially when those beliefs may differ from reality? This question lies at the heart of understanding the Theory of Mind (ToM) capabilities of LMs. We analyze LMs' ability to reason about characters' beliefs using causal mediation and abstraction. We construct a dataset, CausalToM, consisting of simple stories where two characters independently change the state of two objects, potentially unaware of each other's actions. Our investigation uncovers a pervasive algorithmic pattern that we call a lookback mechanism, which enables the LM to recall important information when it becomes necessary. The LM binds each character-object-state triple together by co-locating their reference information, represented as Ordering IDs (OIs), in low-rank subspaces of the state token's residual stream. When asked about a character's beliefs regarding the state of an object, the binding lookback retrieves the correct state OI and then the answer lookback retrieves the corresponding state token. When we introduce text specifying that one character is (not) visible to the other, we find that the LM first generates a visibility ID encoding the relation between the observing and the observed character OIs. In a visibility lookback, this ID is used to retrieve information about the observed character and update the observing character's beliefs. Our work provides insights into belief tracking mechanisms, taking a step toward reverse-engineering ToM reasoning in LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。