arXiv:2511.00739cs.AIcs.LG2025-11被引 7

从CPU视角剖析智能体AI执行瓶颈,提出两项调度优化提升性能。

Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective

  • 从CPU角度分析智能体AI任务的运行瓶颈,识别资源分配不均问题。
  • COMB和MAS优化使同构负载延迟降低1.7倍,异构负载尾部延迟降2.49倍。
  • 适合关注AI服务系统性能调优、资源调度的研究者与工程师。

智能体AI服务将单体大模型推理转变为可自主规划、调用工具、推理并动态适应的问题求解器。由于任务需求多样,该服务严重依赖异构的CPU-GPU系统,多数外部工具由CPU执行或调度。为深入理解其作用,本文首次从被忽视的CPU中心视角,对智能体AI工作负载引入的系统瓶颈进行刻画与分析。首先进行编译时特征分析,选取代表性负载以覆盖算法多样性;随后在两种硬件系统上开展运行时分析,评估端到端延迟与吞吐量,定位架构瓶颈。基于洞察,提出两项调度优化:1. 面向同构负载的CPU感知重叠微批处理(COMB),2. 面向异构负载的混合智能体调度(MAS)。二者均旨在提升CPU-GPU并发利用率,缓解异构执行中的资源倾斜。在两套硬件上的实验表明,COMB在独立同构负载下可降低最高1.7倍的P50延迟,同构开环负载下服务/总延迟分别降低3.9倍/1.8倍;异构开环负载下,少数请求类型的总延迟在P50/P90分位点分别降低2.37倍/2.49倍。

原文摘要 · Abstract (English)

Agentic AI serving converts monolithic LLM-based inference to autonomous problem-solvers that can plan, call tools, perform reasoning, and adapt on the fly. Due to diverse task execution need, such serving heavily rely on heterogeneous CPU-GPU systems with majority of the external tools responsible for agentic capability, either run on or are orchestrated by the CPU. Towards having a deeper understanding of its role, this paper aims to characterize and analyze the system bottlenecks introduced by agentic AI workloads from a largely overlooked CPU-centric perspective. We first present a compile-time characterization of agentic AI execution and choose representative workloads to capture the algorithmic diversity. We then perform runtime characterization of the representative workloads analyzing the end-to-end latency and throughput on two different hardware systems to isolate respective architectural bottlenecks. Based on the insights on the bottlenecks, we finally present two scheduling optimizations, namely, 1. CPU-Aware Overlapped Micro-Batching (COMB) and 2. Mixed Agentic Scheduling (MAS) on homogeneous and heterogeneous agentic workloads, respectively. In specific, these methods optimize for improved CPU-GPU concurrent utilization while reducing skewed resource allocation for heterogeneous execution. Experimental evaluations on the two hardware systems demonstrate the efficacy of COMB in yielding up to 1.7x lower P50 latency in standalone homogeneous workload execution and up to 3.9x/1.8x lower service/total latency under homogeneous open-loop load. Additionally, for heterogeneous open-loop load, MAS can reduce the total latency for minority request-type by up to 2.37x/2.49x at P50/P90 percentile.

智能体AI调度优化性能分析异构系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。