揭示大模型如何用双槽机制跟踪多个实体及其关系
Slot Machines: How LLMs Keep Track of Multiple Entities
- 通过多槽探测分离单个令牌中的当前与前一个实体信息
- 仅当前实体槽用于事实检索，前一实体槽支持关系推理
- 发现模型实际使用的信息少于激活中蕴含的，适合研究认知机制
语言模型需将实体与其属性绑定，并在上下文中维护多个此类关系。我们研究多个实体在词元位置上的表征方式，以及单个词元能否承载多个实体的绑定信息。提出一种多槽探测方法，可解耦单个词元的残差流激活，恢复当前实体和前一实体的信息。这两类信息分别编码在独立且基本正交的“当前实体”和“前一实体”槽中。分析表明二者功能不同：当前实体槽与前一实体槽协同支持关系推理，如实体级归纳（“故事中谁跟在爱丽丝之后？”）和相邻实体间的冲突检测；但只有当前实体槽被用于显式事实检索问题（“故事中有人高吗？”“高的人是谁？”），尽管答案也可从前一实体槽线性解码。开放权重模型在强制单个词元承载两个主语-动词-宾语绑定的语法任务中表现接近随机水平（例如：“爱丽丝准备，鲍勃食用食物。”）。而近期前沿模型能正确解析此类结构，暗示其发展出更复杂的绑定策略。总体结果揭示了激活中可用信息与模型实际使用信息之间的差距，表明当前/前一实体槽结构是同时维持双重视角行为（如阿谀奉承和欺骗）的自然基础。
原文摘要 · Abstract (English)
Language models must bind entities to the attributes they possess and maintain several such binding relationships within a context. We study how multiple entities are represented across token positions and whether single tokens can carry bindings for more than one entity. We introduce a multi-slot probing approach that disentangles a single token's residual stream activation to recover information about both the currently described entity and the immediately preceding one. These two kinds of information are encoded in separate and largely orthogonal "current-entity" and "prior-entity" slots. We analyze the functional roles of these slots and find that they serve different purposes. In tandem with the current-entity slot, the prior-entity slot supports relational inferences, such as entity-level induction ("who came after Alice in the story?") and conflict detection between adjacent entities. However, only the current-entity slot is used for explicit factual retrieval questions ("Is anyone in the story tall?" "What is the tall entity's name?") despite these answers being linearly decodable from the prior-entity slot too. Consistent with this limitation, open-weight models perform near chance accuracy at processing syntax that forces two subject-verb-object bindings on a single token (e.g., "Alice prepares and Bob consumes food.") Interestingly, recent frontier models can parse this properly, suggesting they may have developed more sophisticated binding strategies. Overall, our results expose a gap between information that is available in activations and information the model actually uses, and suggest that the current/prior-entity slot structure is a natural substrate for behaviors that require holding two perspectives at once, such as sycophancy and deception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。