构建自动化运维框架，让智能体系统更稳定可靠。
Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems
- 提出六阶段自动化运维流程，覆盖观察到优化的全链路
- 针对开发者、测试、SRE等角色设计差异化支持机制
- 通过自动分析与反馈实现智能体系统的自适应进化
大型语言模型正越来越多地应用于智能体系统——由多个相互协作、基于LLM的智能体组成的复杂自适应工作流系统，具备记忆、工具调用和动态规划能力。这类系统虽带来强大功能，却引入了源于概率推理、状态演化和执行路径不稳定的独特不确定性。传统软件可观测性手段难以应对。本文提出AgentOps框架，旨在全面实现对智能体AI系统的可观测、分析、优化与自动化。我们识别出开发、测试、站点可靠性工程师（SRE）及业务用户四类角色在系统生命周期中各阶段的不同需求，并提出六阶段自动化运维管道：行为观测、指标采集、问题检测、根因分析、优化建议与运行时自动化。强调自动化并非消除不确定性，而是通过有效管理来确保系统的安全、自适应与高效运行。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed within agentic systems - collections of interacting, LLM-powered agents that execute complex, adaptive workflows using memory, tools, and dynamic planning. While enabling powerful new capabilities, these systems also introduce unique forms of uncertainty stemming from probabilistic reasoning, evolving memory states, and fluid execution paths. Traditional software observability and operations practices fall short in addressing these challenges. This paper presents our vision of AgentOps: a comprehensive framework for observing, analyzing, optimizing, and automating operation of agentic AI systems. We identify distinct needs across four key roles - developers, testers, site reliability engineers (SREs), and business users - each of whom engages with the system at different points in its lifecycle. We present the AgentOps Automation Pipeline, a six-stage process encompassing behavior observation, metric collection, issue detection, root cause analysis, optimized recommendations, and runtime automation. Throughout, we emphasize the critical role of automation in managing uncertainty and enabling self-improving AI systems - not by eliminating uncertainty, but by taming it to ensure safe, adaptive, and effective operation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。