arXiv:2604.16682cs.DCcs.AI2026-04被引 2

KAIROS通过感知代理上下文,动态调节算力与调度,显著降低智能体推理功耗。

KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving

  • 基于代理上下文动态调节GPU频率、并发数和请求分配
  • 实测平均省电27%(最高39.8%),且不牺牲性能目标
  • 适合大规模智能体服务场景,尤其对内存敏感的工程任务

功耗已成为AI推理的核心瓶颈。随着智能体型AI成为主要负载,传统功耗管理技术因仅针对单轮大模型服务而显得不足。我们分析发现,智能体服务具有根本差异:每个请求携带长期演化上下文,且在工具交互中不断变化;降低GPU频率会导致系统进入抖动状态,加剧内存压力,反而损害性能与能效。为此,我们提出KAIROS——一种面向智能体推理的上下文感知功耗优化系统。该系统将代理上下文作为核心控制信号,联合调控GPU频率、实例级并发度及多实例请求部署。这使得在内存充足时可节能,同时避免抖动并维持性能目标。KAIROS以代理为粒度跟踪请求,根据上下文增长与进度自适应调整局部控制策略,并跨实例调度以协同提升能效与内存稳定性。在多样化的软件与数据工程智能体任务上评估,其平均功耗降低27%(最高达39.8%),且满足性能要求。

原文摘要 · Abstract (English)

Power has become a central bottleneck for AI inference. This problem is becoming more urgent as agentic AI emerges as a major workload class, yet prior power-management techniques focus almost entirely on single-turn LLM serving. Our analysis shows that agentic serving behaves fundamentally differently: each request carries long-lived context that evolves across tool-interleaved turns, and lowering GPU frequency can push the system into a thrashing regime where memory pressure sharply worsens both performance and power efficiency. These observations show that power optimization for agentic serving requires rethinking. We present KAIROS, a context-aware power optimization system for agentic AI serving. KAIROS uses agent context as a first-class control signal to jointly manage GPU frequency, per-instance concurrency, and multi-instance request placement. This enables KAIROS to save power when memory headroom exists while avoiding thrashing and preserving performance targets. At a high level, KAIROS tracks requests at agent granularity, adapts local control to context growth and agent progress, and routes agents across instances to jointly improve power efficiency and memory stability. Evaluated across diverse software and data engineering agentic tasks, KAIROS achieves an average of 27% (up to 39.8%) power reduction while meeting the performance targets.

智能体功耗优化推理服务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。