让大模型在社交推理游戏中基于关系信念做决策，提升行为一致性。
MARBO: Relational Belief Grounding for LLM Agents in Social Deduction Games
- 用关系信念指导决策与发言，避免无依据的策略
- 在典型社交推理游戏中，小型模型表现超越现有基线
- 适合研究大模型推理一致性与游戏智能体设计的学者
社交推理游戏（SDGs）要求智能体在信息不完整的情况下，通过维护对隐藏身份和团队关系的信念进行推理。尽管近期基于大模型的智能体通过提示工程和偏好优化提升了游戏表现，但往往未将行为与发言显式地建立在可靠信念之上，导致策略不一致，尤其在小型模型中更为明显。本文提出多智能体关系信念优化（MARBO），一种基于信念的偏好优化框架，利用关系信念引导战略决策与游戏发言。MARBO仅在行为由可信关系信念支持且带来有利社交结果时提供偏好反馈，从而在不确定性下促进更一致的学习。在代表性社交推理游戏上的实验表明，MARBO使小型大模型智能体持续优于现有基线。代码已公开于 https://github.com/PleaseTakemeAway/MARBO。
原文摘要 · Abstract (English)
Social deduction games (SDGs) require agents to reason under partial observability by maintaining relational beliefs about hidden roles and team alignments. While recent LLM-agent approaches improve gameplay through prompting and preference optimization, they often optimize actions and in-game speech without explicitly grounding them in such beliefs. This frequently leads to strategically inconsistent behavior, especially for compact LLM agents. We introduce Multi-Agent Relational Belief Optimization (MARBO), a belief-grounded preference optimization framework that leverages relational beliefs to guide strategic decisions and in-game speech. MARBO provides preference feedback only when behaviors are supported by reliable relational beliefs and lead to strategically favorable social outcomes, encouraging more consistent learning under uncertainty. Experiments on representative SDGs show that MARBO enables compact LLM agents to consistently outperform existing baselines. The Code is available on https://github.com/PleaseTakemeAway/MARBO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。