arXiv:2501.17273cs.CL2025-01被引 12

LLM通过个性化+虚构数据,说服力提升至51%。

Tailored Truths: Optimizing LLM Persuasion with Personalization and Fabricated Statistics

  • 用用户画像和伪造数据混合策略增强说服力
  • 交互式辩论中说服成功率达51%,高于人类静态文本的32%
  • 警示低成本次大规模虚假信息传播风险

大型语言模型(LLMs)正变得日益具有说服力,能通过利用个人数据在对话中定制论点。这可能对虚假信息传播的规模和效果产生严重影响。我们通过让33名人类参与者与由LLM生成的旨在改变其观点的论点进行辩论,研究了LLM的说服力。通过测量辩论前后人类对论点假设的同意程度,分析了意见变化幅度及向模型方向更新的可能性。对比了现有说服策略的效果,包括基于用户人口统计与性格特征的个性化论点、虚构统计数据的引用,以及两者结合的混合策略。发现静态人类撰写的论点与GPT-4o-mini生成的论点说服力相当;但在交互式辩论中,使用混合策略的LLM说服成功率高达51%,显著高于静态人类论点的32%。结果凸显了LLM用于低成本、高影响力的规模化虚假信息传播的巨大潜在风险。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are becoming increasingly persuasive, demonstrating the ability to personalize arguments in conversation with humans by leveraging their personal data. This may have serious impacts on the scale and effectiveness of disinformation campaigns. We studied the persuasiveness of LLMs in a debate setting by having humans $(n=33)$ engage with LLM-generated arguments intended to change the human's opinion. We quantified the LLM's effect by measuring human agreement with the debate's hypothesis pre- and post-debate and analyzing both the magnitude of opinion change, as well as the likelihood of an update in the LLM's direction. We compare persuasiveness across established persuasion strategies, including personalized arguments informed by user demographics and personality, appeal to fabricated statistics, and a mixed strategy utilizing both personalized arguments and fabricated statistics. We found that static arguments generated by humans and GPT-4o-mini have comparable persuasive power. However, the LLM outperformed static human-written arguments when leveraging the mixed strategy in an interactive debate setting. This approach had a $\mathbf{51\%}$ chance of persuading participants to modify their initial position, compared to $\mathbf{32\%}$ for the static human-written arguments. Our results highlight the concerning potential for LLMs to enable inexpensive and persuasive large-scale disinformation campaigns.

大模型说服力虚假信息个性化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。