重新审视大模型理论心理能力评估，揭示评测标准混乱问题
Re-evaluating Theory of Mind evaluation in large language models
- 从认知科学出发，区分应评估行为还是背后推理机制
- 指出现有评测混入非纯理论心理内容，导致结果不可靠
- 建议未来研究结合语用交流，深化对智能本质理解
关于大语言模型（LLMs）是否具备理论心理（ToM）——即推断他人心理状态的能力——的问题引发了科学界和公众的广泛关注。然而，目前关于该问题的证据不一，评估方法的快速增长并未带来共识。本文受认知科学启发，重新审视大模型中理论心理评估的现状。我们提出，判断模型是否具备理论心理存在分歧的主要原因在于：未明确模型应模仿人类行为，还是再现其背后的计算机制。同时，我们指出当前评估方式可能偏离了“纯”理论心理能力的测量，这也加剧了混淆。最后，我们讨论了未来研究方向，包括理论心理与语用交流的关系，这有助于推进对人工系统及人类认知的理解。
原文摘要 · Abstract (English)
The question of whether large language models (LLMs) possess Theory of Mind (ToM) -- often defined as the ability to reason about others' mental states -- has sparked significant scientific and public interest. However, the evidence as to whether LLMs possess ToM is mixed, and the recent growth in evaluations has not resulted in a convergence. Here, we take inspiration from cognitive science to re-evaluate the state of ToM evaluation in LLMs. We argue that a major reason for the disagreement on whether LLMs have ToM is a lack of clarity on whether models should be expected to match human behaviors, or the computations underlying those behaviors. We also highlight ways in which current evaluations may be deviating from "pure" measurements of ToM abilities, which also contributes to the confusion. We conclude by discussing several directions for future research, including the relationship between ToM and pragmatic communication, which could advance our understanding of artificial systems as well as human cognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。