给大模型做心理测评,看它像不像人。
Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications
- 用心理学工具评估大模型的人格特质
- 发现部分模型在特定提示下人格稳定,但整体差异大
- 适合研究可信AI、人机交互的学者参考
随着大语言模型(LLMs)越来越多地应用于以人类为中心的任务,评估其心理特征对于理解其社会影响并确保可信的AI对齐至关重要。现有综述虽涵盖部分内容,但对多样化的心理测试、专为大模型设计的心理数据集,以及具有心理特质的大模型应用等关键领域缺乏系统讨论。本文系统回顾了将心理学理论应用于大模型的六个关键维度:(1) 评估工具;(2) 大模型专用数据集;(3) 评价指标(一致性与稳定性);(4) 实证发现;(5) 人格模拟方法;(6) 基于大模型的行为模拟。分析表明,尽管某些大模型在特定提示方案下表现出可重复的人格模式,但在不同任务和设置中仍存在显著变异性。针对心理工具与大模型能力之间的不匹配、评估实践不一致等方法论挑战,本研究旨在提出未来更可解释、更鲁棒、更具泛化性的大模型心理评估框架发展方向。
原文摘要 · Abstract (English)
As large language models (LLMs) are increasingly used in human-centered tasks, assessing their psychological traits is crucial for understanding their social impact and ensuring trustworthy AI alignment. While existing reviews have covered some aspects of related research, several important areas have not been systematically discussed, including detailed discussions of diverse psychological tests, LLM-specific psychological datasets, and the applications of LLMs with psychological traits. To address this gap, we systematically review six key dimensions of applying psychological theories to LLMs: (1) assessment tools; (2) LLM-specific datasets; (3) evaluation metrics (consistency and stability); (4) empirical findings; (5) personality simulation methods; and (6) LLM-based behavior simulation. Our analysis highlights both the strengths and limitations of current methods. While some LLMs exhibit reproducible personality patterns under specific prompting schemes, significant variability remains across tasks and settings. Recognizing methodological challenges such as mismatches between psychological tools and LLMs' capabilities, as well as inconsistencies in evaluation practices, this study aims to propose future directions for developing more interpretable, robust, and generalizable psychological assessment frameworks for LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。