用多个模型集体讨论,减少大模型幻觉和偏见。
Council Mode: A Heterogeneous Multi-Agent Consensus Framework for Reducing LLM Hallucination and Bias

- 让不同顶尖大模型并行回答问题,再由专门模型整合共识。
- 幻觉率降低41.7%,真理问答得分提升7.5分,多领域推理质量达95.4%。
- 适合对准确率要求高、可接受额外计算成本的场景。
大型语言模型虽能力强大,但常出现事实错误(幻觉)和系统性偏见,尤其在混合专家(MoE)架构中更为明显。为此,我们提出Council Mode,一种异构多智能体共识框架。该方法将查询智能分配至多个前沿大模型并行处理,并通过专用共识模型合成输出。流程分为三阶段:查询复杂度智能分诊、跨模型并行生成、结构化整合以识别一致、分歧与独特发现。在无网络的受控环境下评估显示,于1,200样本的HaluEval子集上,幻觉率相对降低41.7%;在TruthfulQA上比最优单模型提升7.5分;在自建的MDR-500多领域推理基准上,质量得分为95.4%,优于最佳单模型9.2分。框架在基于评分的评估中表现出更低的偏见方差。成本效益分析表明,推理开销增加4.2倍,适用于错误代价高于额外成本的高精度场景。结果表明,结构化多智能体共识是提升大模型内容可靠性与事实性的有效路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated advanced capabilities but often suffer from factual inaccuracies (hallucinations) and systematic biases. These issues, sometimes amplified in specific architectures like Mixture-of-Experts (MoE) which motivate our work, pose risks for reliable deployment. To address these challenges, we propose the Council Mode, a multi-agent consensus framework. Our approach dispatches queries to multiple heterogeneous frontier LLMs in parallel and synthesizes their outputs using a dedicated consensus model. The pipeline consists of three phases: an intelligent triage for query complexity, parallel generation across diverse models, and a structured synthesis that identifies agreement, disagreement, and unique findings. In our evaluation, conducted under controlled no-web settings, the Council Mode achieved a 41.7% relative reduction in hallucination rates on a 1,200-sample HaluEval subset and a 7.5-point improvement on TruthfulQA compared to the top-performing individual model. On our curated MDR-500 multi-domain reasoning benchmark, the Council Mode achieved a Quality Score of 95.4%, representing a 9.2-point improvement over the best individual model. The framework also exhibited lower measured bias variance under our rubric-based evaluation protocol. We provide a cost-effectiveness analysis showing that the framework incurs a 4.2x token-cost overhead, making it most suitable for accuracy-prioritized applications where the cost of errors exceeds the added inference cost. These findings suggest that structured multi-agent consensus is a promising direction for enhancing the reliability and factual grounding of LLM-generated content.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。