用大模型直接预测民意分布,比反复模拟个体更准更省
Characterizing the ability of LLMs to recapitulate Americans' distributional responses to public opinion polling questions across political issues
- 直接提示大模型预测回答分布,而非模拟个体
- 准确率高于传统模拟方法,成本降低显著
- 性能可预测,适合提前评估模型表现
传统政治议题民意调查因成本上升、回应率下降及关键人群覆盖不足而日益难以实施。随着研究人员和民调机构寻求替代方案,大型语言模型在辅助人口研究方面展现出潜力。本文提出并实现了一种新框架,通过直接提示大模型预测多选题的响应分布,以预估公众意见。基于美国全国性高质量民调数据集——合作选举研究(Cooperative Election Study),我们评估该框架在不同人口统计学特征与多种议题下的表现,并与此前通过反复查询大模型模拟个体响应的方法进行比较。结果表明,所提框架在多数情况下准确率更高,且成本显著降低。此外,该框架在不同群体与问题间的性能变化更具系统性与可预测性,使民调者仅凭查询前信息即可有效预判模型表现。
原文摘要 · Abstract (English)
Traditional survey-based political issue polling is becoming less tractable due to increasing costs and risk of bias associated with growing non-response rates and declining coverage of key demographic groups. With researchers and pollsters seeking alternatives, Large Language Models have drawn attention for their potential to augment human population studies in polling contexts. We propose and implement a new framework for anticipating human responses on multiple-choice political issue polling questions by directly prompting an LLM to predict a distribution of responses. By comparison to a large and high quality issue poll of the US population, the Cooperative Election Study, we evaluate how the accuracy of this framework varies across a range of demographics and questions on a variety of topics, as well as how this framework compares to previously proposed frameworks where LLMs are repeatedly queried to simulate individual respondents. We find the proposed framework consistently exhibits more accurate predictions than individual querying at significantly lower cost. In addition, we find the performance of the proposed framework varies much more systematically and predictably across demographics and questions, making it possible for those performing AI polling to better anticipate model performance using only information available before a query is issued.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。