智能触发多智能体辩论,高效提升大模型推理准确率。
iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference
- 基于自我批判内容提取41个语言特征,判断是否需触发辩论
- 减少92%令牌消耗,准确率最高提升13.5%
- 无需针对测试数据调优,适合高并发推理场景
大型语言模型(LLM)代理系统发展迅速,其在零样本场景下具备强大泛化能力。为提升复杂任务上的推理与准确性,多智能体辩论(MAD)框架应运而生,通过多个LLM代理结构化辩论促进多样化推理。然而,对每个查询都触发MAD效率低下,因带来巨大计算(令牌)开销,甚至可能推翻正确单智能体答案导致准确率下降。为此,我们提出智能多智能体辩论(iMAD),一种令牌高效的框架,仅在初始回答可能错误时才触发辩论。iMAD通过学习可泛化的模型行为进行精准辩论决策:首先引导单智能体生成结构化自我批判响应,从中提取41个可解释的语言与语义特征以捕捉犹豫线索;随后使用轻量级辩论决策分类器,基于提出的FocusCal损失进行训练,实现无需测试集特定调优的稳健决策。在六个(视觉)问答数据集上对五种竞争基线的广泛实验表明,iMAD显著降低令牌使用量(最高达92%),同时提升最终答案准确率(最高13.5%)。
原文摘要 · Abstract (English)
Large Language Model (LLM) agent systems have advanced rapidly, driven by their strong generalization in zero-shot settings. To further enhance reasoning and accuracy on complex tasks, Multi-Agent Debate (MAD) has emerged as a promising framework that engages multiple LLM agents in structured debates to encourage diverse reasoning. However, triggering MAD for every query is inefficient, as it incurs substantial computational (token) cost and may even degrade accuracy by overturning correct single-agent answers. To address these limitations, we propose intelligent Multi-Agent Debate (iMAD), a token-efficient framework that selectively triggers MAD only when it is likely to be beneficial (i.e., correcting an initially wrong answer). To achieve this goal, iMAD learns generalizable model behaviors to make accurate debate decisions. Specifically, iMAD first prompts a single agent to produce a structured self-critique response, from which we extract 41 interpretable linguistic and semantic features capturing hesitation cues. Then, iMAD uses a lightweight debate-decision classifier, trained using our proposed FocusCal loss, to determine whether to trigger MAD, enabling robust debate decisions without test dataset-specific tuning. Through extensive experiments using six (visual) question answering datasets against five competitive baselines, we have shown that iMAD significantly reduces token usage (by up to 92%) while also improving final answer accuracy (by up to 13.5%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。