大模型能预测对话中人的偏见决策,还复现了认知负荷的影响。
Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings
- 用对话复杂度模拟认知负荷,测试大模型预测人类偏见决策能力。
- 对话越复杂,框架效应和现状偏差越明显,大模型准确捕捉该现象。
- GPT-4表现最佳,适合用于设计自适应对话系统。
我们研究大语言模型(LLMs)是否能预测对话场景中的人类偏见决策,以及其预测是否反映认知偏见及其在认知负荷下的变化。一项预注册研究(N = 1,648)中,参与者通过聊天机器人完成六个经典决策任务,对话复杂度各异。参与者表现出框架效应和现状偏差两种典型认知偏见。对话复杂度增加导致心理负担上升,且该认知负荷显著放大了偏见效应,验证了负荷-偏见交互作用。随后评估了GPT-4、GPT-5及开源模型在提供人口统计信息与历史对话的基础上预测个体决策的能力。尽管结果因任务而异,但结合对话上下文的预测在多个关键场景中显著更准确。更重要的是,大模型预测再现了人类观察到的偏见模式和负荷-偏见交互。所有模型中,GPT-4家族始终最贴近人类行为,在预测准确性和偏见模式拟合度上均优于GPT-5与开源模型。这些发现推动了对大模型作为人类决策模拟工具的理解,并为自适应对话代理的设计提供依据。
原文摘要 · Abstract (English)
We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load. In a pre-registered study (N = 1,648), participants completed six classic decision-making tasks via a chatbot with dialogues of varying complexity. Participants exhibited two well-documented cognitive biases: the Framing Effect and the Status Quo Bias. Increased dialogue complexity resulted in participants reporting higher mental demand. This increase in cognitive load selectively, but significantly, increased the effect of the biases, demonstrating the load-bias interaction. We then evaluated whether LLMs (GPT-4, GPT-5, and open-source models) could predict individual decisions given demographic information and prior dialogue. While results were mixed across choice problems, LLM predictions that incorporated dialogue context were significantly more accurate in several key scenarios. Importantly, their predictions reproduced the same bias patterns and load-bias interactions observed in humans. Across all models tested, the GPT-4 family consistently aligned with human behavior, outperforming GPT-5 and open-source models in both predictive accuracy and fidelity to human-like bias patterns. These findings advance our understanding of LLMs as tools for simulating human decision-making and inform the design of conversational agents that adapt to user biases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。