构建可追踪信念变化的对话框架,揭示大模型说服人类的真实过程。
A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

- 用贝叶斯网络模拟人类信念动态,实现多轮对话中的认知真实更新
- 发现人类信念更新分两类模式,且大模型在多模态下均具说服力
- 提出过程级评估方法,比传统前后对比更精准反映说服机制
大型语言模型可在高风险领域改变人类信念,但多数说服研究仅依赖前后信念变化测量,无法捕捉对话中信念的演变路径。本文提出PERSUASIONTRACE框架,基于网络实验平台,支持多轮说服研究与过程级评估:记录人类或模拟目标的多轮信念报告,标注说服者话语的修辞维度(逻辑/情感/可信度),并通过与真实人类信念动态的吻合度评估模拟器。实验发现,人类目标可分为两类多轮信念更新模式,且对修辞策略敏感;大模型在通用与个性化话题、文本与音频模态、多轮交互中均具说服力。以往研究多使用普通提示的LLM模拟人类目标,但本文表明其无法复制真实信念动态。为此引入贝叶斯网络模拟目标,显式维护随时间演化的潜在信念状态,使每条说服信息产生认知上合理的信念更新。在人类相似性评估中,该贝叶斯目标得分81(人类参考为80),而基线LLM目标仅得64。PERSUASIONTRACE将说服评估从终点变化转向过程保真度,为科学分析和安全优化说服系统提供更强基础。
原文摘要 · Abstract (English)
Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint measures identify whether persuasion occurred, yet miss where and how beliefs moved within a dialogue. We present PERSUASIONTRACE, a framework for studying persuasion in human-LLM interaction. Built on a web-based experimental platform, PERSUASIONTRACE contributes a tool for multi-turn persuasion studies and a process-level evaluation protocol: it records multi-turn belief reports from human or simulated targets of persuasion, annotates persuader turns with rhetorical dimensions (logos/pathos/ethos), and evaluates simulators by fidelity to real human belief dynamics. Using this framework, we find that human targets group into two clusters of multi-turn belief updates and exhibit susceptibility to rhetorical strategies, and that LLMs are persuasive across generic and personalized topics, text and audio modalities, and multi-turn interactions. Prior work has chiefly used vanilla-prompted LLMs to simulate human targets, but we show that these simulators fail to replicate human belief dynamics. We introduce a Bayesian-network simulated target that maintains an explicit latent belief state over time so each persuader message yields cognitively realistic belief updates. In human-likeness evaluation, our Bayesian target scores near a human reference (81 vs 80), while baseline LLM targets score substantially lower (64). PERSUASIONTRACE reframes persuasion evaluation from endpoint movement alone to process fidelity, providing a stronger basis for scientific analysis and safer optimization of persuasive systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。