GPT-4模拟人类回答社会调查时表现出矛盾的迎合倾向。
Exploring Social Desirability Response Bias in Large Language Models: Evidence from GPT-4 Simulations
- 用GPT-4模拟四国人群，测试其是否产生迎合社会期望的偏差
- 承诺声明使迎合度上升但公民参与度下降，结果相互矛盾
- 适合研究人类与模型自身偏见，尤其关注社会调查中的响应偏差
大型语言模型（LLMs）被用于模拟社会调查中的人类回应，但尚不清楚它们是否会像人类一样产生社会期望响应偏差（SDR）。为探究此问题，研究将GPT-4分配至四个社会群体角色，使用2022年盖洛普世界民意调查数据。在有或无承诺声明的条件下进行提问，以诱导SDR。结果显示：承诺声明虽提升了SDR指数得分，表明存在迎合倾向，却同时降低了公民参与度得分，呈现相反趋势。此外，发现人口统计特征与SDR得分相关，且承诺声明对GPT-4预测性能影响有限。该研究揭示了利用LLMs探索人类及模型自身偏见的潜在路径。
原文摘要 · Abstract (English)
Large language models (LLMs) are employed to simulate human-like responses in social surveys, yet it remains unclear if they develop biases like social desirability response (SDR) bias. To investigate this, GPT-4 was assigned personas from four societies, using data from the 2022 Gallup World Poll. These synthetic samples were then prompted with or without a commitment statement intended to induce SDR. The results were mixed. While the commitment statement increased SDR index scores, suggesting SDR bias, it reduced civic engagement scores, indicating an opposite trend. Additional findings revealed demographic associations with SDR scores and showed that the commitment statement had limited impact on GPT-4's predictive performance. The study underscores potential avenues for using LLMs to investigate biases in both humans and LLMs themselves.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。