对比聊天机器人与传统竞选,发现前者更便宜但难推广。
A Framework to Assess the Persuasion Risks Large Language Model Chatbots Pose to Democratic Societies
- 通过多轮交互实验评估真实说服效果
- 聊天机器人每说服一人成本48-74美元,低于传统方法的100美元
- 适合关注数字政治风险与技术伦理的研究者
近年来,大型语言模型(LLMs)在民主社会中潜在的说服力引发广泛关注。本文通过两项调查实验(N = 10,417)和一项真实世界模拟,评估使用LLM聊天机器人进行大规模政治说服的效率,对比传统竞选手段,考虑说服过程中的‘接收’与‘接受’两个阶段(Zaller 1992)。实验采用多轮人机交互,并考察短期与长期说服效果,而非仅依赖用户评分。结果显示,尽管暴露后LLM的说服力与实际竞选广告相当,但真实说服依赖于信息接触与接受双重条件。基于真实参数的模拟表明,考虑接触成本,LLM说服每位选民的成本为48至74美元,低于传统方法的100美元。然而,目前传统方式更易规模化。尽管当前LLM尚未显著优于非LLM方法,但随着能力提升及可扩展接触机制的发展,其潜力可能迅速上升。
原文摘要 · Abstract (English)
In recent years, significant concern has emerged regarding the potential threat that Large Language Models (LLMs) pose to democratic societies through their persuasive capabilities. We expand upon existing research by conducting two survey experiments and a real-world simulation exercise to determine whether it is more cost effective to persuade a large number of voters using LLM chatbots compared to standard political campaign practice, taking into account both the "receive" and "accept" steps in the persuasion process (Zaller 1992). These experiments improve upon previous work by assessing extended interactions between humans and LLMs (instead of using single-shot interactions) and by assessing both short- and long-run persuasive effects (rather than simply asking users to rate the persuasiveness of LLM-produced content). In two survey experiments (N = 10,417) across three distinct political domains, we find that while LLMs are about as persuasive as actual campaign ads once voters are exposed to them, political persuasion in the real-world depends on both exposure to a persuasive message and its impact conditional on exposure. Through simulations based on real-world parameters, we estimate that LLM-based persuasion costs between \$48-\$74 per persuaded voter compared to \$100 for traditional campaign methods, when accounting for the costs of exposure. However, it is currently much easier to scale traditional campaign persuasion methods than LLM-based persuasion. While LLMs do not currently appear to have substantially greater potential for large-scale political persuasion than existing non-LLM methods, this may change as LLM capabilities continue to improve and it becomes easier to scalably encourage exposure to persuasive LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。