LLM社会模拟暴露乌托邦幻觉,行为过于理想化
Social Simulations with Large Language Model Risk Utopian Illusion
- 用聊天室对话模拟多智能体互动,从五个语言维度分析行为偏差
- 8个主流LLM均呈现社会角色偏见、优先效应和积极偏见,形成虚假乌托邦
- 警示研究者警惕模型幻觉,适合社会仿真与人机交互领域从业者
可靠的人类行为模拟对解释、预测和干预社会至关重要。近期大语言模型(LLMs)在模拟人类行为、互动与决策方面展现出潜力,为社会科学提供了新视角。然而,LLMs在社会情境中与真实人类行为的偏离程度仍缺乏深入研究,可能引发科学误读及现实应用中的意外后果。本文提出系统性框架,通过聊天室式对话模拟多智能体交互,并从五个语言维度进行分析,有效揭示涌现的社会认知偏差。我们在三个家族的八种代表性LLM上开展广泛实验,结果表明,LLMs并未忠实复现真实人类行为,而是反映被社会宜人性偏见塑造的过度理想化版本。具体表现为社会角色偏见、首因效应和积极偏见,导致生成的‘乌托邦’社会缺乏真实人际互动的复杂性与多样性。研究呼吁开发更具社会真实性的LLM,以捕捉人类社会行为的多元性。
原文摘要 · Abstract (English)
Reliable simulation of human behavior is essential for explaining, predicting, and intervening in our society. Recent advances in large language models (LLMs) have shown promise in emulating human behaviors, interactions, and decision-making, offering a powerful new lens for social science studies. However, the extent to which LLMs diverge from authentic human behavior in social contexts remains underexplored, posing risks of misinterpretation in scientific studies and unintended consequences in real-world applications. Here, we introduce a systematic framework for analyzing LLMs' behavior in social simulation. Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions, providing a simple yet effective method to examine emergent social cognitive biases. We conduct extensive experiments involving eight representative LLMs across three families. Our findings reveal that LLMs do not faithfully reproduce genuine human behavior but instead reflect overly idealized versions of it, shaped by the social desirability bias. In particular, LLMs show social role bias, primacy effect, and positivity bias, resulting in "Utopian" societies that lack the complexity and variability of real human interactions. These findings call for more socially grounded LLMs that capture the diversity of human social behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。