对话AI的说服力主要来自训练和提示方法，而非模型规模或个性化。
The Levers of Political Persuasion with Conversational AI
- 通过提示工程和后训练提升说服力，最高可增51%和27%
- 越有说服力的AI，其陈述越不准确，事实错误率上升
- 适合关注AI伦理、传播效果与可信度的研究者
针对对话式AI可能对人类信念产生空前影响的担忧，我们在三组大规模实验（总样本量N=76,977）中部署了19个大语言模型（包括部分专门优化说服力的模型），评估其在707个政治议题上的说服力。随后检查了466,769条模型生成陈述的事实准确性。结果显示，当前及近未来AI的说服力更多源于后训练与提示策略——分别使说服力提升最高达51%和27%——而非模型规模或个性化。这些方法通过快速调用并战略性使用信息增强说服力，但令人震惊的是，说服力增强的同时，事实准确性系统性下降。
原文摘要 · Abstract (English)
There are widespread fears that conversational AI could soon exert unprecedented influence over human beliefs. Here, in three large-scale experiments (N=76,977), we deployed 19 LLMs-including some post-trained explicitly for persuasion-to evaluate their persuasiveness on 707 political issues. We then checked the factual accuracy of 466,769 resulting LLM claims. Contrary to popular concerns, we show that the persuasive power of current and near-future AI is likely to stem more from post-training and prompting methods-which boosted persuasiveness by as much as 51% and 27% respectively-than from personalization or increasing model scale. We further show that these methods increased persuasion by exploiting LLMs' unique ability to rapidly access and strategically deploy information and that, strikingly, where they increased AI persuasiveness they also systematically decreased factual accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。