动态路由筛选模型，大幅降低大模型协作的算力与延迟开销。
RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents
- 先用轻量评分器预判模型表现，筛出高潜力候选集，无需全量推理。
- 通过自评与互评机制修正评分，实现无需额外推理的后验优化。
- 兼顾性能、成本与延迟，在大规模模型池中降本超90%、提速63.6%。
Mixture-of-Agents (MoA) 通过分层协作提升 LLM 性能，但其密集拓扑导致高昂成本与延迟。现有方法依赖 LLM 判官在推理后过滤结果，仍需全部模型完成推理，无法有效降本。同时缺乏模型选择标准，且在大规模模型池中，全量推理成本过高，可能超出上下文长度限制。为此，我们提出 RouteMoA，一种高效混合代理框架，支持动态路由。它使用轻量级评分器，基于查询预测粗粒度性能，无须推理即可缩小候选范围。随后，一组判官通过基于已有模型输出的轻量级自评与互评，进行后验修正，无需额外推理。最终，模型排序机制综合考量性能、成本与延迟，选出最优模型。RouteMoA 在不同任务和模型池规模下均优于 MoA，大规模模型池中成本降低 89.8%，延迟减少 63.6%。
原文摘要 · Abstract (English)
Mixture-of-Agents (MoA) improves LLM performance through layered collaboration, but its dense topology raises costs and latency. Existing methods employ LLM judges to filter responses, yet still require all models to perform inference before judging, failing to cut costs effectively. They also lack model selection criteria and struggle with large model pools, where full inference is costly and can exceed context limits. To address this, we propose RouteMoA, an efficient mixture-of-agents framework with dynamic routing. It employs a lightweight scorer to perform initial screening by predicting coarse-grained performance from the query, narrowing candidates to a high-potential subset without inference. A mixture of judges then refines these scores through lightweight self- and cross-assessment based on existing model outputs, providing posterior correction without additional inference. Finally, a model ranking mechanism selects models by balancing performance, cost, and latency. RouteMoA outperforms MoA across varying tasks and model pool sizes, reducing cost by 89.8% and latency by 63.6% in the large-scale model pool.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。