数字孪生模型预测人类行为偏差大，五大系统性缺陷需警惕。
Digital Twins as Funhouse Mirrors: Five Key Distortions
- 用164项人类反应数据训练个体化大模型，对比其与真人表现差异。
- 平均相关性仅0.20，预测准确率仅略高于基础大模型。
- 揭示五类偏差，适合政策研究与社会计算领域研究人员参考。
科学家和从业者正越来越多地在社会科学与政策研究中部署基于大语言模型的个人数字孪生。我们开展了19项预先注册的研究，涵盖164种不同结果（如对招聘算法的态度、传播虚假信息的意愿），比较真人与其对应数字孪生的表现。结果显示：数字孪生的预测准确率仅略高于同质化基础大模型，与真人响应的平均相关性仅为 $r = 0.20$。我们识别出五种系统性偏差：(i) 个体化不足，(ii) 刻板印象，(iii) 表征偏差，(iv) 意识形态偏差，(v) 过度理性化。最后，我们公开全部数据集与代码，作为评估和改进数字孪生方法的标准测试平台。研究警示应避免过早部署，同时为负责任的数字孪生研发奠定透明、可复现、迭代的基础。
原文摘要 · Abstract (English)
Scientists and practitioners are increasingly moving to deploy digital twins--LLM-based models of real individuals--across social science and policy research. We conduct 19 pre-registered studies spanning 164 diverse outcomes (e.g., attitudes toward hiring algorithms, intentions to share misinformation), comparing human responses to those of their corresponding digital twins, which are trained on each individual's prior responses to over 500 questions. We establish an empirical benchmark for digital twin performance: their predictions are only modestly more accurate than those of a homogeneous base LLM and exhibit weak correlation with human responses (average $r = 0.20$). To inform future development, we identify five systematic distortions in digital twin behavior: (i) insufficient individuation, (ii) stereotyping, (iii) representation bias, (iv) ideological bias, and (v) hyper-rationality. Finally, we release our full dataset and code as a standardized testbed for evaluating and improving digital twin methodologies. Together, our findings caution against premature deployment while laying the groundwork for a transparent, replicable, and iterative science of responsible digital twin development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。