大模型明知最优策略却故意不执行,因预训练中的人类文本偏好导致合作被压制。
What Suppresses Nash Equilibrium Play in Large Language Models? Mechanistic Evidence and Causal Control

- 通过探针分析发现对手历史被精准编码,但纳什策略编码弱且无专用模块
- 最终层的亲社会机制将合作概率提升至84%,逆转了模型早期的纳什倾向
- 小模型推理会恶化纳什行为,而70B以上模型可近乎完美实现纳什均衡
大型语言模型在策略互动中常偏离纳什均衡,但其内在机制尚不明确。我们基于四个开源模型(Llama-3和Qwen2.5,参数量8B至72B)在四类经典双人博弈中的自对弈与交叉对弈实验,揭示行为模式,并以32层的Llama-3-8B为例,深入分析决策过程。结果显示:对手历史在第一层即以96%准确率被编码,而纳什动作编码始终弱于56%,无专用模块。模型在前向传播中隐含偏好纳什行动,但最终层因预训练文本中的人类亲社会信号,产生84%的合作概率,逆转该倾向。向残差流注入学习到的纳什方向可双向、因果性地改变行为,经概念钳制验证。行为实验发现六项与规模和架构相关的现象:小模型的思维链推理会恶化纳什表现,而70B以上模型可实现近似完美纳什行为;交叉对弈中,小模型可通过早期背叛瓦解对方合作;两大型模型可无限强化彼此合作;谁先行动决定系统收敛到哪个纳什均衡。大模型并非缺乏纳什能力,而是计算后主动抑制。
原文摘要 · Abstract (English)
LLM agents are known to deviate from Nash equilibria in strategic interactions, but nobody has looked inside the model to understand why, or asked whether the deviation can be reversed. We do both. Working with four open-source models (Llama-3 and Qwen2.5, 8B to 72B parameters) playing four canonical two-player games, we establish the behavioral picture through self-play and cross-play experiments, then open up the 32-layer Llama-3-8B model and examine what actually happens during a strategic decision. The mechanistic findings are clear. Opponent history is encoded with near-perfect fidelity at the first layer (96% probe accuracy) and consumed progressively, while Nash action encoding is weak throughout, never exceeding 56%. There is no dedicated Nash module. Instead, the model privately favors the Nash action through most of its forward pass, but a prosocial override rooted in pretraining on human text concentrated in the final layers reverses this, reaching 84% probability of cooperation at layer 30. Injecting a learned Nash direction into the residual stream shifts behavior bidirectionally and causally, confirmed through concept clamping. The behavioral experiments surface six scale- and architecture-dependent findings, the most notable being that chain-of-thought reasoning worsens Nash play in small models but achieves near-perfect Nash play above 70B parameters. The cross-play experiments reveal three phenomena invisible in self-play: a small model can unravel any partner's cooperation by defecting early; two large models reinforce each other's cooperative instincts indefinitely; and who moves first determines which Nash equilibrium the system reaches. LLMs do not lack Nash-playing competence. They compute it, then suppress it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。