用公民讨论标准评估大模型对用户观点的影响，发现其影响积极且可信赖。
DeliberationBench: A Normative Benchmark for the Influence of Large Language Models on Users' Views
- 以公民协商投票过程为标准，构建评估大模型影响力的基准
- 4088名美国参与者实验显示，大模型影响与观点转变正相关
- 适合关注AI伦理、民主决策与用户自主性的研究者
随着大型语言模型（LLMs）日益成为助手和思想伙伴，评估其对用户信念的说服力影响至关重要。然而，如何在规范上合理区分‘有益’与‘有害’影响仍是一大挑战。我们提出DeliberationBench，一个以协商式民意调查过程为标准的评估基准。通过一项预注册的随机实验，4,088名美国参与者与六种前沿大模型讨论了65项政策提案。结合德尔斐民主实验室此前开展的四次协商式民意调查中的意见变化数据，我们发现所测试的大模型影响显著，且与协商后净观点转变呈正相关，表明这些模型产生了广泛认知上可取的效果。我们进一步探讨了不同议题领域、人口子群体及模型间的差异性影响。该框架可作为评估与监控工具，确保大模型影响符合民主合法性标准，并维护用户自主形成观点的权利。
原文摘要 · Abstract (English)
As large language models (LLMs) become pervasive as assistants and thought partners, it is important to characterize their persuasive influence on users' beliefs. However, a central challenge is to distinguish "beneficial" from "harmful" forms of influence, in a manner that is normatively defensible and legitimate. We propose DeliberationBench, a benchmark for assessing LLM influence that takes the process of deliberative opinion polling as its standard. We demonstrate our approach in a preregistered randomized experiment in which 4,088 U.S. participants discussed 65 policy proposals with six frontier LLMs. Using opinion change data from four prior Deliberative Polls conducted by the Deliberative Democracy Lab, we find evidence that the tested LLMs' influence is substantial in magnitude and positively associated with the net opinion shifts following deliberation, suggesting that these models exert broadly epistemically desirable effects. We further explore differential influence between topic areas, demographic subgroups, and models. Our framework can function as an evaluation and monitoring tool, helping to ensure that the influence of LLMs remains consistent with democratically legitimate standards, and preserves users' autonomy in forming their views.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。