arXiv:2604.26963cs.OScs.DC2026-04被引 2

MARS优化异构系统中智能体的资源调度,显著降低延迟。

MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems

论文配图:MARS: Efficient, Adaptive Co-Scheduling for Heterogeneous Agentic Systems
图 1 · 摘自论文原文
  • 通过统一信息流实现GPU与CPU资源全局可见,解耦任务准入与执行
  • 端到端延迟降低5.94倍,系统吞吐量接近最大值
  • 适合部署复杂智能体系统的生产环境,尤其适用于代码生成等场景

大型语言模型(LLMs)正越来越多地作为自主智能体的核心执行单元,而非单纯的文本生成工具。智能体工作负载导致执行模式从单轮推理转向多轮模型-工具循环,并从仅用GPU的聊天规模执行,演变为需在存储库规模下进行GPU-CPU协同部署的执行模式。因此,协调智能体执行中的异构资源需求已成为关键系统挑战。我们设计并实现了MARS——一个高效且自适应的协同调度系统,可在耦合的GPU-CPU资源压力下全局协调异构智能体工作负载。通过统一信息流建立对GPU推理与CPU工具执行的全局可视性,外部控制平面将任务准入与执行解耦,防止资源超分配。内部以智能体为中心的调度器通过优先处理延迟敏感的延续,并仅在热重启能带来延迟收益时才保留KV缓存状态,从而最小化端到端关键路径。评估显示,MARS可将端到端延迟降低最高达5.94倍,同时维持近乎最大系统吞吐量。我们将MARS集成至OpenHands代码智能体框架的后端服务中,实测其将端到端任务完成时间加速最高达1.87倍。源代码已公开于https://github.com/Afterglow231/MARS_preview。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators. Agentic workloads induce a temporal shift from single-turn inference to multi-turn LLM-tool loops, and a spatial shift from chat-scale, GPU-only execution to repository-scale, GPU-CPU co-located execution. Consequently, coordinating heterogeneous resource demands of agentic execution has emerged as a critical system challenge. We design and implement MARS, an efficient and adaptive co-scheduling system that globally coordinates heterogeneous agentic workloads under coupled GPU-CPU resource pressure. By establishing holistic visibility across GPU inference and CPU tool execution via a unified information stream, an external control plane in MARS decouples admission from execution to prevent heterogeneous resource oversubscription. An internal agent-centric scheduler further minimizes the end-to-end critical path by prioritizing latency-sensitive continuations and adaptively retaining KV cache state only when warm resumption yields a latency benefit. Our evaluations show that MARS reduces end-to-end latency by up to 5.94x while maintaining nearly maximal system throughput. We further integrate MARS as the serving backend for the OpenHands coding agent framework, demonstrating its real-world effectiveness by accelerating end-to-end task completion time by up to 1.87x. Our source code is publicly available at https://github.com/Afterglow231/MARS_preview .

智能体系统资源调度异构计算延迟优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。