用调查数据训练的AI代理比仅靠人口统计更准确预测退休态度。
From Demographics to Survey Anchors: Evaluating LLM Agents for Modeling Retirement Attitudes

- 用调查回答构建代理,比仅用人口统计更贴近真实
- 仅用人统计的代理会过度偏向平均值且忽略错误回答
- 适合研究退休规划行为的学者和政策制定者
大型语言模型(LLM)代理可能用于预测人类对调查的回应。当前常见方法仅使用人口统计信息(如国家、年龄、性别、职业、收入、教育程度和婚姻状况)。本文对比了仅基于人口统计的代理与基于更广泛领域内调查回答的代理在预测跨国家、多学科的《欧洲健康、老龄化与退休调查》(SHARE)中五个个人金融相关变量时的准确性。结果发现:相比调查锚定代理,仅用人统计的代理(1)表现出中心趋势偏差,答案过度趋近于总体均值;(2)过于精确,未能复现人类受访者常见的错误回答和“不知道”选项。通过复现既有退休规划研究中的分层回归分析,进一步验证:仅用人统计的代理虽能重现风险偏好、未来时间视角和退休规划知识对储蓄的独立影响,但只有调查锚定代理成功再现这三者间的交互作用。这提示仅凭人口统计定义LLM代理存在局限。
原文摘要 · Abstract (English)
Large language models (LLM) agents may offer tools to predict human responses to surveys. A common technique for defining these agents uses only demographics, for example country, age, gender, employment status, income, education and marital status. We compare the predictive accuracy of demographic agents to that of survey agents defined with a larger set of in-domain survey responses. We test both approaches in predicting responses to the multidisciplinary, cross-national Survey of Health, Ageing and Retirement in Europe (SHARE), focusing on five variables from three policy-relevant constructs around personal finance. In these three constructs, we observe that, compared to survey agents trained on broader data, demographics-only agents (1) exhibited a central tendency bias, skewing answers toward population means, and (2) were unrealistically accurate, failing to reproduce the incorrect answers and "don't know" responses typical of human respondents. These performance differences are further substantiated through the replication of a hierarchical regression analysis from prior retirement planning research. Agents based solely on demographic information reproduce the outcome that financial risk tolerance, future time perspective, and knowledge of retirement planning each are predictive of retirement savings. However, only the survey-anchored agents succeed in reproducing the interaction among these three factors. These findings suggest caution in using only demographics to define LLM agents for predicting survey responses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。