arXiv:2502.19130cs.MAcs.AI2025-02ACL被引 74

对比七种决策协议,发现投票更适合推理,共识更适合知识任务

Voting or Consensus? Decision-Making in Multi-Agent Debate

  • 固定其他变量,只改变决策协议,系统评估其影响
  • 投票使推理任务提升13.2%,共识使知识任务提升2.8%
  • 提出新方法AAD和CI,最多提升7.4%性能,适合优化多智能体协作

多智能体辩论的成功很大程度上取决于参数的合理选择,其中决策协议对最终模型答案有显著影响。由于多数研究同时改变多个讨论参数,系统比较决策协议极为困难,目前尚不清楚决策方式如何影响不同任务。本文系统评估了七种决策协议(如多数投票、一致共识),仅改变决策协议一个变量,分析其对智能体协作的影响,并测量在知识与推理任务中的差异。结果表明,相比其他协议,投票协议在推理任务中提升13.2%,共识协议在知识任务中提升2.8%。增加智能体数量可提升性能,但讨论轮次过多会降低表现。为提升答案多样性以改进决策,提出两种新方法:全智能体起草(AAD)和集体改进(CI),分别带来最高3.3%和7.4%的任务性能提升。本工作揭示了决策机制在多智能体辩论中的关键作用,超越单纯扩展规模。

原文摘要 · Abstract (English)

Much of the success of multi-agent debates depends on carefully choosing the right parameters. The decision-making protocol stands out as it can highly impact final model answers, depending on how decisions are reached. Systematic comparison of decision protocols is difficult because many studies alter multiple discussion parameters beyond the protocol. So far, it has been largely unknown how decision-making influences different tasks. This work systematically evaluates the impact of seven decision protocols (e.g., majority voting, unanimity consensus). We change only one variable at a time - the decision protocol - to analyze how different methods affect the collaboration between agents and measure differences in knowledge and reasoning tasks. Our results show that voting protocols improve performance by 13.2% in reasoning tasks and consensus protocols by 2.8% in knowledge tasks compared to other decision protocols. Increasing the number of agents improves performance, while more discussion rounds before voting reduce it. To improve decision-making by increasing answer diversity, we propose two new methods, All-Agents Drafting (AAD) and Collective Improvement (CI). Our methods improve task performance by up to 3.3% with AAD and up to 7.4% with CI. This work demonstrates the importance of decision-making in multi-agent debates beyond scaling.

多智能体决策协议辩论系统推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。