MOSAIC通过智能调度与动态聚合，让多专家大模型推理更快更省显存。
MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency
- 用数学规划优化专家分配和任务分发，减少资源浪费。
- 在4卡系统上，推理阶段提速2.5倍，聚合阶段快4.23倍。
- 适合需要高并发、低延迟的多模型协同推理场景。
混合专家（MoA）系统通过将每个查询路由至多个专家大模型并融合输出来提升推理准确性。然而，在有限的GPU资源下高效执行此类工作负载存在瓶颈：基于技能的路由导致专家负载不均，而指令微调模型与长推理模型结合又造成生成长度极端波动。传统调度策略因此出现显著的GPU空闲和吞吐量崩溃。本文提出MOSAIC调度框架：首先构建基于整数线性规划（ILP）的调度器，利用离线性能数据联合优化专家部署与每工作节点的提示分配，复制推理专家于多个工作节点，同时固定轻量级模型；其次采用置信度感知的自适应聚合机制，利用专家间一致性跳过耗时的最终聚合器。在4-GPU系统中，MOSAIC相比基线调度器实现最高2.5倍的专家阶段加速、4.23倍的聚合阶段加速，以及1.7~2.3倍的端到端加速，准确率仅下降0.1个百分点以内。
原文摘要 · Abstract (English)
Mixture-of-Agents (MoA) systems improve reasoning accuracy by routing each query to multiple expert LLMs and aggregating their outputs. Efficiently executing this workload on limited GPU resources has bottlenecks. Skill-based routing creates skewed expert demand, and combining instruction-tuned LLMs with long-reasoning models results in extreme variability in generation lengths. Consequently, traditional scheduling strategies suffer from significant GPU idling and throughput collapse due to load imbalances. We present MOSAIC, a scheduling framework to accelerate MoA workloads. First, we formulate an Integer Linear Program (ILP) based scheduler that jointly optimizes expert placement and per-worker prompt assignment from offline-profiled costs, replicating reasoning experts across workers while pinning lightweight ones. Second, MOSAIC uses confidence-aware adaptive aggregation, leveraging inter-expert agreement to bypass the heavy final aggregator LLM for consensus queries. In our 4-GPU system, MOSAIC achieves up to 2.5x expert-stage, 4.23x aggregator-stage and 1.7~2.3x end-to-end speedups over the baseline scheduler, while matching accuracy within 0.1pp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。