用概率模型提升大模型社交推理能力，首次让语言代理击败人类玩家。
Bayesian Social Deduction with Graph-Informed Language Models
- 将信念推断交给结构化概率模型，大模型专注语言理解与交互。
- 在多人对战中表现媲美超大模型，击败人类玩家胜率67%。
- 适合研究社交推理、人机协作的AI开发者和研究人员。
社交推理——从其他智能体的部分行为推断其不可观测的信念与意图——仍是大语言模型（LLMs）面临的挑战。我们评估了当前推理型语言模型在社交推理游戏Avalon中的表现，发现尽管最大模型表现优异，但需大量测试时推理，且在压缩为实时可用的小模型后性能急剧下降。为此，我们提出一种混合推理框架：将信念推断外置于结构化概率模型，同时利用大模型进行语言理解与交互。该方法在智能体-智能体对战中达到与大型模型相当的性能，且首次在受控实验中让语言代理击败人类玩家，胜率达67%，并获得比基准模型和人类队友更高的定性评价。代码、模型及数据集已开源，详见https://camp-lab-purdue.github.io/bayesian-social-deduction/
原文摘要 · Abstract (English)
Social reasoning - inferring unobservable beliefs and intentions from partial observations of other agents - remains a challenging task for large language models (LLMs). We evaluate the limits of current reasoning language models in the social deduction game Avalon and find that while the largest models demonstrate strong performance, they require extensive test-time inference and degrade sharply when distilled to smaller, real-time-capable variants. To address this, we introduce a hybrid reasoning framework that externalizes belief inference to a structured probabilistic model, while using an LLM for language understanding and interaction. Our approach achieves competitive performance with much larger models in Agent-Agent play and, notably, is the first language agent to defeat human players in a controlled study - achieving a 67% win rate and receiving higher qualitative ratings than both reasoning baselines and human teammates. We release code, models, and a dataset to support future work on social reasoning in LLM agents, which can be found at https://camp-lab-purdue.github.io/bayesian-social-deduction/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。