让大模型学会可审计的谈判信念推理,提升决策透明度。
Distilling Bayesian Belief States into Language Models for Auditable Negotiation

- 用贝叶斯框架建模对手优先级顺序,动态更新信念分布。
- 8B小模型保持0.114的信念准确性,优于基准参考值0.139。
- 支持信念轨迹分析与干预,便于调试和验证谈判逻辑。
谈判代理需推断对方价值、随对话轮次更新信念,并在不确定性下决策。端到端大语言模型(LLM)可模仿谈判对话,但其对手信念通常隐含且难以检视。本文提出BOND(贝叶斯对手信念谈判蒸馏)框架,包含一个基于LLM的贝叶斯教师模型,对六种可能的对手优先级排序进行评分,更新后验分布,并据此进行菜单式决策;同时训练一个更小的8B学生语言模型,输出谈判动作及归一化的后验信念作为标记文本。在CaSiNo谈判数据集上,BOND超越当前最优水平,平均贝叶斯得分(Brier score)为0.085。蒸馏后的学生模型仍保留较强信念信号,达到0.114的贝叶斯得分,低于均匀六排序参考值5/36(约0.139)。相比70B结构化思维链基线,8B学生模型在信念校准方面表现更优。通过后验轨迹、信念-策略误差分解与后验前缀干预,进一步展示了可审计性。诊断结果表明,蒸馏比因果信念控制更能保留可测量的信念报告,使弱信念-动作耦合显现而非隐藏。
原文摘要 · Abstract (English)
Negotiation agents must infer what their counterpart values, update those beliefs over dialogue turns, and choose actions under uncertainty. End-to-end large language models (LLMs) can imitate negotiation dialogue, but their opponent beliefs are usually implicit and difficult to inspect. We propose BOND (Bayesian Opponent-belief Negotiation Distillation), a framework for auditable negotiation. BOND consists of an LLM-based Bayesian teacher that scores dialogue contexts against the six possible opponent priority orderings, updates a posterior over those orderings, and uses the posterior for menu-based decision making, as well as a smaller 8B student language model that emits both negotiation actions and normalized posterior beliefs as tagged text. In the CaSiNo negotiation dataset, BOND outperforms the state-of-the-art and achieves mean Brier score 0.085 over opponent-priority posteriors. The distilled student preserves much of this belief signal, achieving Brier 0.114, below the uniform six-ordering reference of 5/36, approximately 0.139. Compared with a 70B structured-CoT baseline, the significantly smaller 8B student model yields substantially better elicited posterior calibration. We further showcase auditability through posterior trajectories, belief-versus-policy error decomposition, and posterior-prefix interventions. These diagnostics reveal that distillation preserves a scoreable belief report more strongly than causal belief-conditioned control, making weak belief-action coupling visible, not hidden.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。