arXiv:2606.14350cs.DCcs.AI2026-06

用多模型协作系统替代单模型,实现更低延迟与成本。

Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systems

  • 将多个模型分布式编排,通过可配置工作流提升灵活性
  • 性能接近单模型,但延迟降低60%,成本减少71%
  • 适合需平衡精度、速度与开销的AI系统设计者

人工智能系统需满足精度、延迟和成本等服务级别目标。现有以模型为中心的方法在设计时选定单一模型,对所有输入采用相同计算,无法跨组件分解任务,且知识固定于训练阶段,运行时易导致性能下降与成本上升。由于模型是主要设计变量,它决定了系统行为,使运营目标与单一设计决策紧密耦合。为解决此问题,需从模型中心转向系统中心设计。复合型AI系统通过显式控制逻辑,将多个模型、算法与工具协同为分布式系统,实现这一转变。其性能取决于工作流拓扑、各任务分配的模型及运行时参数。本文提出一种设计方法,沿工作流拓扑与配置选择两个维度组织设计空间,识别出八种设计模式,每种整合技术以应对单体部署的特定局限。通过三个案例验证,复合型配置在精度上仅比单体模型低2.5至4个百分点,同时延迟降低最多60%,成本减少最多71%。我们表明,模型选择与参数配置共同决定系统性能,但设计空间随工作流组合模式与组件增加呈组合爆炸增长。因此,我们识别出五个开放挑战,勾勒出从手动配置原型迈向自动发现并维持SLO合规的复合与分布式AI系统的路线图。

原文摘要 · Abstract (English)

Artificial Intelligence (AI) systems must typically satisfy service-level objectives including accuracy, latency, and cost. The prevailing model-centric approaches select a monolithic model at design time and apply identical computation regardless of input difficulty, cannot decompose tasks across specialized components, and have knowledge that is fixed at training time. During runtime, this can lead to performance degradation and increasing costs. Because the model is the main design variable, it determines the majority of system behavior, coupling operational objectives to a single design-time choice. Addressing these limitations requires shifting from model-centric to system-centric design. Compound AI systems realize this shift by orchestrating multiple models, algorithms, and tools as distributed AI systems through explicit control logic. The performance of such systems depends on their workflow topology, the models assigned to each task, and the parameters governing runtime behavior. We present a design methodology that organizes this space along two dimensions, workflow topology and configuration selection, and identifies eight design patterns, each consolidating techniques to address a specific limitation of monolithic deployment. We validate our methodology through three case studies. Across our case studies, Compound AI configurations approach accuracy of monolithic models within 2.5 to 4 percentage points while reducing latency by up to 60% and cost by up to 71%. We show that model selection and parameter configuration jointly determine system performance, but the resulting design space grows combinatorially, as workflows compose more patterns and components. Thus, we identify five open challenges that define a roadmap from manually configured prototypes towards systems that automatically discover and maintain SLO-compliance in Compound and Distributed AI systems.

系统设计分布式AI性能优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。