用多智能体辩论降低大模型流水线的误判和成本
CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascades
- 在每级升级边界引入轻量级智能体辩论，只对不确定问题触发
- 在五个基准上最高提升26.75%，比固定策略准确率高52.33%
- 适合需要动态控本又追求高精度的部署场景
级联大模型系统通过协调不同规模的模型与人工专家，在准确性、成本和不确定性拒答之间取得平衡。然而，单模型层级在处理模糊查询时常因信心不足导致过早升级至更昂贵的模型或专家，造成计算效率低下。CascadeDebate 在每级升级边界引入多智能体辩论机制，仅当置信度低时激活轻量级代理集群，通过共识内部解决歧义，避免不必要的升级。统一架构在不同模型尺度间交替进行单模型推理与选择性多智能体辩论，最终以人工专家为兜底。该设计根据查询难度动态调整推理时计算量。在涵盖科学、医学和通用知识的五个基准上，相比强单模型级联和独立多智能体系统，性能最高提升26.75%。在线阈值优化器显著增强效果，相较固定策略准确率提升20.98%至52.33%，实现对真实分布的弹性适应。
原文摘要 · Abstract (English)
Cascaded LLM systems coordinate models of varying sizes with human experts to balance accuracy, cost, and abstention under uncertainty. However, single-model tiers at each stage often struggle with ambiguous queries, triggering premature escalations to costlier models or experts due to under-confidence and inefficient compute scaling. CascadeDebate addresses this gap by inserting multi-agent deliberation directly at each tier's escalation boundary. Confidence-based routers activate lightweight agent ensembles only for uncertain cases, enabling consensus-driven resolution of ambiguities internally without invoking higher-cost upgrades. Our unified architecture alternates single-model inference with selective multi-agent deliberation across model scales, culminating in human experts as the final fallback. This design scales test-time compute dynamically according to query difficulty. Across five benchmarks spanning science, medicine, and general knowledge, CascadeDebate outperforms strong single-model cascades and standalone multi-agent systems by up to 26.75 percent. An online threshold optimizer proves essential, boosting accuracy by 20.98 to 52.33 percent relative improvement over fixed policies and enabling elastic adaptation to real-world distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。