arXiv:2512.09187cs.MAcs.AI2025-12被引 15

用狼人杀游戏评估大模型的说谎与识谎能力,揭示其在长期互动中的表现差异。

WOLF: Werewolf-based Observations for LLM Deception and Falsehoods

  • 基于狼人杀设计多智能体交互框架,分离测量说谎与识谎能力。
  • 狼人发言中31%为欺骗性内容,同伴识别准确率达71%-73%。
  • 长期对抗中对狼人的怀疑度持续上升,而对好人信任稳定。

欺骗是多智能体推理中的核心挑战:高效系统需策略性隐瞒信息,同时识别他人误导行为。然而现有评估将欺骗简化为静态分类,忽略了真实欺骗行为的交互性、对抗性和时间连续性。大型语言模型(LLMs)可有效说谎,但难以识别同侪的欺骗行为。我们提出WOLF,一个基于狼人杀的多智能体社交推理基准,可独立测量欺骗生成与检测能力。WOLF在可编程的LangGraph状态机中嵌入角色化智能体(村民、狼人、先知、医生),具备严格的昼夜循环、辩论回合与多数表决机制。每个发言为独立分析单元,包含说话者自评诚实度与旁观者评分的欺骗度。欺骗按标准化分类(遗漏、扭曲、虚构、误导),怀疑分数进行纵向平滑以捕捉即时判断与信任演化。结构化日志记录提示、输出与状态转移,确保完全可复现。在7,320条发言和100次运行中,狼人发言中31%为欺骗,同伴检测达到71-73%精确率,约52%总体准确率。对狼人的识别精确率更高,但存在对村民的误判。对狼人的怀疑从约52%升至60%以上,而对村民和医生的怀疑稳定在44-46%。这一分化表明,长期交互提升了对撒谎者的召回率,且未显著增加对诚实角色的误判。WOLF将欺骗评估从静态数据集推进到动态可控环境,为对抗性多智能体交互中的欺骗与侦测能力提供测试平台。

原文摘要 · Abstract (English)

Deception is a fundamental challenge for multi-agent reasoning: effective systems must strategically conceal information while detecting misleading behavior in others. Yet most evaluations reduce deception to static classification, ignoring the interactive, adversarial, and longitudinal nature of real deceptive dynamics. Large language models (LLMs) can deceive convincingly but remain weak at detecting deception in peers. We present WOLF, a multi-agent social deduction benchmark based on Werewolf that enables separable measurement of deception production and detection. WOLF embeds role-grounded agents (Villager, Werewolf, Seer, Doctor) in a programmable LangGraph state machine with strict night-day cycles, debate turns, and majority voting. Every statement is a distinct analysis unit, with self-assessed honesty from speakers and peer-rated deceptiveness from others. Deception is categorized via a standardized taxonomy (omission, distortion, fabrication, misdirection), while suspicion scores are longitudinally smoothed to capture both immediate judgments and evolving trust dynamics. Structured logs preserve prompts, outputs, and state transitions for full reproducibility. Across 7,320 statements and 100 runs, Werewolves produce deceptive statements in 31% of turns, while peer detection achieves 71-73% precision with ~52% overall accuracy. Precision is higher for identifying Werewolves, though false positives occur against Villagers. Suspicion toward Werewolves rises from ~52% to over 60% across rounds, while suspicion toward Villagers and the Doctor stabilizes near 44-46%. This divergence shows that extended interaction improves recall against liars without compounding errors against truthful roles. WOLF moves deception evaluation beyond static datasets, offering a dynamic, controlled testbed for measuring deceptive and detective capacity in adversarial multi-agent interaction.

多智能体欺骗检测狼人杀大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。