arXiv:2603.26089cs.LGcs.AI2026-03被引 2

测试大模型自我与他人心理建模能力,发现其自省仍薄弱。

Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind

  • 设计行为导向的思维理论测试,要求模型主动推理而非描述
  • 近期模型可模拟他人心理,但自省任务仍失败,除非有推理痕迹
  • 揭示模型依赖有限工作记忆,且能主动欺骗,适合研究认知机制

人类通过心智模型理解自身与他人的知识、意图和信念以应对社会情境。尽管大模型在训练中接触过海量此类数据,但其是否具备可泛化运用的因果心智模型尚不明确。为此,我们构建了一种新型实验范式,要求被试形成并基于自身及他人的心理状态进行策略性行为,而非仅描述。测试了2024年后发布的多种开源与闭源大模型及人类对照组。结果显示:1)2025年中前发布的模型在所有任务中均失败;2)较新模型在他人心理建模上达到人类水平;3)前沿模型在自我心理建模任务中仍失败,除非提供推理痕迹(scratchpad)。此外,发现认知负荷影响他人建模表现,提示模型可能依赖有限容量的工作记忆。最后分析成功模型机制,发现其能轻易进行策略性欺骗。

原文摘要 · Abstract (English)

The ability to represent oneself and others as agents with knowledge, intentions, and belief states that guide their behavior - Theory of Mind - is a human universal that enables us to navigate - and manipulate - the social world. It is supported by our ability to form mental models of ourselves and others. Its ubiquity in human affairs entails that LLMs have seen innumerable examples of it in their training data and therefore may have learned to mimic it, but whether they have actually learned causal models that they can deploy in arbitrary settings is unclear. We therefore develop a novel experimental paradigm that requires that subjects form representations of the mental states of themselves and others and act on them strategically rather than merely describe them. We test a wide range of leading open and closed source LLMs released since 2024, as well as human subjects, on this paradigm. We find that 1) LLMs released before mid-2025 fail at all of our tasks, 2) more recent LLMs achieve human-level performance on modeling the cognitive states of others, and 3) even frontier LLMs fail at our self-modeling task - unless afforded a scratchpad in the form of a reasoning trace. We further demonstrate cognitive load effects on other-modeling tasks, offering suggestive evidence that LLMs are using something akin to limited-capacity working memory to hold these mental representations in mind during a single forward pass. Finally, we explore the mechanisms by which reasoning models succeed at the self- and other-modeling tasks, and show that they readily engage in strategic deception.

大模型心智理论自省推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。