用大模型生成的虚拟人重做性格测试，发现能模拟出部分人类人格结构。
Applying Psychometrics to Large Language Model Simulated Populations: Recreating the HEXACO Personality Inventory Experiment with Generative Agents
- 用310个GPT-4虚拟人模拟人格问卷，基于回复做因子分析。
- 结果部分符合HEXACO人格框架，且在同模型内具一致性。
- 不同模型表现差异明显，暴露其人格偏见与局限性。
由大型语言模型驱动的生成式代理通过复杂的自然语言交互展现出类人特征。其基于预设人物背景扮演角色与人格的能力，使其成为社会科学实验中低成本的人类参与者替代方案。本文探讨此类基于人格设定的代理在代表人类群体方面的有效性：我们复现了Ashton、Lee与Goldberg于2004年提出的HEXACO人格量表实验，对310名由GPT-4驱动的代理进行问卷调查，对其回答进行因子分析，并与原始研究结果对比。结果显示：1）从代理的回答中可恢复出一个连贯且可靠的个性结构，显示出与HEXACO框架的部分一致性；2）在足够精心筛选的群体下，该人格维度在GPT-4中具有内部一致性和可靠性；3）跨模型分析揭示人格刻画存在显著差异，表明模型特异性偏差与局限性。本文讨论实验中的实际挑战与考量，为利用生成代理开展社会科学研究提供有益指导，尤其在设计一致、有代表性的代理人格方面。
原文摘要 · Abstract (English)
Generative agents powered by Large Language Models demonstrate human-like characteristics through sophisticated natural language interactions. Their ability to assume roles and personalities based on predefined character biographies has positioned them as cost-effective substitutes for human participants in social science research. This paper explores the validity of such persona-based agents in representing human populations; we recreate the HEXACO personality inventory experiment by surveying 310 GPT-4 powered agents, conducting factor analysis on their responses, and comparing these results to the original findings presented by Ashton, Lee, & Goldberg in 2004. Our results found 1) a coherent and reliable personality structure was recoverable from the agents' responses demonstrating partial alignment to the HEXACO framework. 2) the derived personality dimensions were consistent and reliable within GPT-4, when coupled with a sufficiently curated population, and 3) cross-model analysis revealed variability in personality profiling, suggesting model-specific biases and limitations. We discuss the practical considerations and challenges encountered during the experiment. This study contributes to the ongoing discourse on the potential benefits and limitations of using generative agents in social science research and provides useful guidance on designing consistent and representative agent personas to maximise coverage and representation of human personality traits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。