通过树形路由与动态剪枝，显著降低多智能体推理延迟。
Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap
- 用树状结构替代密集通信，实现稀疏化智能体交互
- 运行时根据语义一致性动态跳过或终止下游智能体调用
- 重叠预填充与解码过程，提升硬件利用率
多智能体（MoA）推理常因智能体间密集通信和硬件利用率低导致服务延迟升高。本文提出一种算法-系统协同设计的推理方案：首先，将密集的智能体交互图替换为分层树状拓扑，引入结构化稀疏性；其次，引入运行时自适应机制，基于中间输出的语义一致性和置信度信号，选择性终止或跳过下游智能体调用；第三，通过依赖相关智能体间的增量预填充与解码重叠，实现流水线执行，提升资源利用率并降低延迟。在多个典型任务上，该方法可将端到端延迟降低高达90%，同时保持与稠密连接基线相当的精度（误差在±1%内），部分场景下甚至提升准确率。
原文摘要 · Abstract (English)
Mixture-of-Agents (MoA) inference can suffer from dense inter-agent communication and low hardware utilization, which jointly inflate serving latency. We present a serving design that targets these bottlenecks through an algorithm-system co-design. First, we replace dense agent interaction graphs with a hierarchical tree topology that induces structured sparsity in inter-agent communication. Second, we introduce a runtime adaptive mechanism that selectively terminates or skips downstream agent invocations using semantic agreement and confidence signals from intermediate outputs. Third, we pipeline agent execution by overlapping incremental prefilling with decoding across dependency-related agents, improving utilization and reducing inference latency. Across representative tasks, this approach substantially reduces end-to-end latency (up to 90%) while maintaining comparable accuracy (within $\pm$1%) relative to dense-connectivity MoA baselines, and can improve accuracy in certain settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。