arXiv:2601.01522cs.AIcs.CL2026-01被引 2

用贝叶斯框架协调多个大模型,降低决策成本并提升公平性。

Bayesian Orchestration of Multi-LLM Agents for Cost-Aware Sequential Decision-Making

  • 将大模型视为似然估计器,通过对比提示获取多模型证据并贝叶斯更新信念
  • 在简历筛选中节省34%总成本,达29.4万美元,同时降低群体差距45%
  • 适合高成本误判场景,如招聘、医疗分诊、反欺诈等需要持续决策的领域

大型语言模型(LLMs)正被用于部署于错误代价不对称的自主决策场景中,如招聘(漏掉人才与浪费面试)、医疗分诊(漏诊紧急情况与过度升级)和欺诈检测(批准欺诈与拒绝合法支付)。当前主流方法仅依赖单个LLM输出状态后验概率,设定置信阈值后执行动作;我们证明此法在序列决策中成本不优。本文提出一种贝叶斯驱动、成本感知的多LLM协同框架,将LLM视为近似似然模型而非分类器。对每个候选状态,通过对比提示获取各模型的似然值,利用稳健统计聚合多个模型结果,并在新证据到来时基于显式先验使用贝叶斯规则更新信念。该方法实现一致信念更新、期望成本最优的动作选择、基于信息价值的合理信息采集,以及通过集成偏差缓解带来的公平性提升。在简历筛选任务中,以每漏聘一人损失40,000美元、每面试1人成本2,500美元、每电话面1人150美元为设定,基于1000份简历,采用五个模型(GPT-4o、Claude 4.5 Sonnet、Gemini Pro、Grok、DeepSeek)的实验表明,总成本减少294,000美元(降幅34%),优于最优单模型基线;同时民主平等性提升45%(最大群体差距从22降至5个百分点)。消融实验显示,51%的节约源于多模型聚合,43%来自序列更新,20%来自分歧触发的信息获取,与正确概率基础的理论优势一致。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly deployed as autonomous decision agents in settings with asymmetric error costs: hiring (missed talent vs wasted interviews), medical triage (missed emergencies vs unnecessary escalation), and fraud detection (approved fraud vs declined legitimate payments). The dominant design queries a single LLM for a posterior over states, thresholds "confidence," and acts; we prove this is inadequate for sequential decisions with costs. We propose a Bayesian, cost-aware multi-LLM orchestration framework that treats LLMs as approximate likelihood models rather than classifiers. For each candidate state, we elicit likelihoods via contrastive prompting, aggregate across diverse models with robust statistics, and update beliefs with Bayes rule under explicit priors as new evidence arrives. This enables coherent belief updating, expected-cost action selection, principled information gathering via value of information, and fairness gains via ensemble bias mitigation. In resume screening with costs of 40000 USD per missed hire, 2500 USD per interview, and 150 USD per phone screen, experiments on 1000 resumes using five LLMs (GPT-4o, Claude 4.5 Sonnet, Gemini Pro, Grok, DeepSeek) reduce total cost by 294000 USD (34 percent) versus the best single-LLM baseline and improve demographic parity by 45 percent (max group gap 22 to 5 percentage points). Ablations attribute 51 percent of savings to multi-LLM aggregation, 43 percent to sequential updating, and 20 percent to disagreement-triggered information gathering, consistent with the theoretical benefits of correct probabilistic foundations.

多智能体贝叶斯推理成本优化公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。