让大模型自由表达,更真实地模拟公众意见。
Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation
- 用自由文本替代选择题,释放大模型的生成能力。
- 能捕捉未预料观点,减少研究者预设偏见。
- 适合社会科学研究者探索复杂民意现象。
大型语言模型(LLMs)正被广泛用于模拟公众意见等社会现象。现有研究多采用多选题或简答形式以方便评分与比较,但这类封闭设计忽略了LLM固有的生成特性。本文主张,在社会模拟中应充分利用自由文本的开放性,以捕捉话题、观点和推理过程。结合数十年调查方法学研究与近期自然语言处理进展,我们论证了开放性对提升测量精度、支持意外观点发现、降低研究者引导偏差的重要性。它还能体现个体表达差异,辅助预测试验,增强方法论实用性。我们呼吁建立新实践与评估框架,充分发挥LLM生成多样性潜力,推动自然语言处理与社会科学的协同创新。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used to simulate public opinion and other social phenomena. Most current studies constrain these simulations to multiple-choice or short-answer formats for ease of scoring and comparison, but such closed designs overlook the inherently generative nature of LLMs. In this position paper, we argue that open-endedness, using free-form text that captures topics, viewpoints, and reasoning processes "in" LLMs, is essential for realistic social simulation. Drawing on decades of survey-methodology research and recent advances in NLP, we argue why this open-endedness is valuable in LLM social simulations, showing how it can improve measurement and design, support exploration of unanticipated views, and reduce researcher-imposed directive bias. It also captures expressiveness and individuality, aids in pretesting, and ultimately enhances methodological utility. We call for novel practices and evaluation frameworks that leverage rather than constrain the open-ended generative diversity of LLMs, creating synergies between NLP and social science.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。