测试大模型模拟特定群体观点分布的能力，发现其描述优于模拟。
Benchmarking Distributional Alignment of Large Language Models
- 构建新数据集，覆盖政治外多领域观点分布
- 大模型能更准确描述而非复制群体观点分布
- 揭示大模型模拟人类群体的局限性，适合研究群体建模者
大语言模型（LMs）日益被用作人类的模拟体，但其在匹配特定人群观点分布（即分布对齐）方面的能力尚不明确。这一概念复杂，因需模拟的属性类型差异显著。现有研究忽视了三个关键变量：问题领域、引导方法和分布表达方式，因此我们构建了一个明确涵盖这些维度的基准。我们扩展了数据集范围，超越政治价值观，建立人类基线，并评估模型在多大程度上可与特定群体的观点分布对齐，以指导此类模拟系统的设计。分析表明，大模型能否以及如何模拟人类仍存开放问题，且其在描述观点分布上优于实际模拟。
原文摘要 · Abstract (English)
Language models (LMs) are increasingly used as simulacra for people, yet their ability to match the distribution of views of a specific demographic group and be \textit{distributionally aligned} remains uncertain. This notion of distributional alignment is complex, as there is significant variation in the types of attributes that are simulated. Prior works have underexplored the role of three critical variables -- the question domain, steering method, and distribution expression method -- which motivates our contribution of a benchmark explicitly addressing these dimensions. We construct a dataset expanding beyond political values, create human baselines for this task, and evaluate the extent to which an LM can align with a particular group's opinion distribution to inform design choices of such simulation systems. Our analysis reveals open problems regarding if, and how, LMs can be used to simulate humans, and that LLMs can more accurately describe the opinion distribution than simulate such distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。