arXiv:2506.04301cs.LGcs.AR2025-06中稿 · publication at the…被引 45

首次系统分析AI代理的运行成本,揭示其高效背后的资源消耗危机。

The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective

  • 从基础设施视角量化分析代理的资源、延迟与能耗
  • 多步推理带来准确率提升,但收益递减且延迟波动大
  • 适合关注模型部署效率与可持续性的研究者和工程师

基于大语言模型(LLM)的AI代理通过动态推理展现出惊人灵活性,能协调外部工具进行多步决策。这种从静态单次推理转向动态多轮工作流的转变,虽提升了任务泛化能力和行为灵活性,但也带来了系统级成本、效率与可持续性问题。本文首次对AI代理进行全面的系统级分析,量化了不同代理设计和测试时扩展策略下的资源消耗、延迟行为、能耗及数据中心级电力需求。研究发现,尽管增加计算量可提升准确率,但边际收益迅速下降,延迟方差扩大,基础设施成本不可持续。通过对代表性代理的细致评估,揭示了代理工作流带来的巨大计算负荷,暴露出潜在的可持续性危机。因此,亟需在代理设计中转向更高效的推理范式,在性能与实际部署约束间取得平衡。

原文摘要 · Abstract (English)

Large-language-model (LLM)-based AI agents have recently showcased impressive versatility by employing dynamic reasoning, an adaptive, multi-step process that coordinates with external tools. This shift from static, single-turn inference to agentic, multi-turn workflows broadens task generalization and behavioral flexibility, but it also introduces serious concerns about system-level cost, efficiency, and sustainability. This paper presents the first comprehensive system-level analysis of AI agents, quantifying their resource usage, latency behavior, energy consumption, and datacenter-wide power consumption demands across diverse agent designs and test-time scaling strategies. We further characterize how AI agent design choices, such as few-shot prompting, reflection depth, and parallel reasoning, impact accuracy-cost tradeoffs. Our findings reveal that while agents improve accuracy with increased compute, they suffer from rapidly diminishing returns, widening latency variance, and unsustainable infrastructure costs. Through detailed evaluation of representative agents, we highlight the profound computational demands introduced by AI agent workflows, uncovering a looming sustainability crisis. These results call for a paradigm shift in agent design toward compute-efficient reasoning, balancing performance with deployability under real-world constraints.

AI代理计算成本系统分析可持续性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。