arXiv:2504.18039cs.AI2025-04中稿 · ACMMM 2025被引 10

让游戏机器人学会读脸识音、揣摩人心,更像真人。

MultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind

  • 融合表情、语音与语言,多模态理解玩家行为
  • 用心理模型预测他人怀疑对象,降低自身被怀疑风险
  • 在真人对战中表现优于传统模型,逼近人类水平

大型语言模型代理在狼人杀等社交推理游戏中展现出强大能力,但现有方法仅依赖文本信息,忽略了人脸表情和语调等关键多模态线索。同时,现有代理主要关注推断其他玩家身份,却未建模他人对自己的认知或对同伴的看法。为此,我们以《一晚终极狼人杀》(One Night Ultimate Werewolf, ONUW)为测试平台,提出MultiMind框架,首次将多模态信息融入社交推理游戏代理。该框架结合面部表情、语音语调与言语内容进行分析,并引入心理理论(Theory of Mind, ToM)模型,动态刻画每位玩家对他人的怀疑程度。通过将此ToM模型与蒙特卡洛树搜索(MCTS)结合,代理能识别出使自身被怀疑度最低的沟通策略。在代理间对抗模拟及真人参与实验中,MultiMind均展现出优越的游戏表现。本工作标志着大模型代理向具备类人社会推理能力迈出关键一步。

原文摘要 · Abstract (English)

Large Language Model (LLM) agents have demonstrated impressive capabilities in social deduction games (SDGs) like Werewolf, where strategic reasoning and social deception are essential. However, current approaches remain limited to textual information, ignoring crucial multimodal cues such as facial expressions and tone of voice that humans naturally use to communicate. Moreover, existing SDG agents primarily focus on inferring other players' identities without modeling how others perceive themselves or fellow players. To address these limitations, we use One Night Ultimate Werewolf (ONUW) as a testbed and present MultiMind, the first framework integrating multimodal information into SDG agents. MultiMind processes facial expressions and vocal tones alongside verbal content, while employing a Theory of Mind (ToM) model to represent each player's suspicion levels toward others. By combining this ToM model with Monte Carlo Tree Search (MCTS), our agent identifies communication strategies that minimize suspicion directed at itself. Through comprehensive evaluation in both agent-versus-agent simulations and studies with human players, we demonstrate MultiMind's superior performance in gameplay. Our work presents a significant advancement toward LLM agents capable of human-like social reasoning across multimodal domains.

狼人杀多模态心理理论智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。