让失败操作变成训练信号,提升AI运维的可靠性与安全。
AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis
- 用策略优化方法从专家经验中学习,不泄露敏感数据。
- 失败轨迹被转为纠错信号,使系统越用越准。
- 适合需要高安全、持续改进的工业级AI运维场景。
大型语言模型(LLM)代理为自动化站点可靠性工程(SRE)提供了有前景的数据驱动路径,但其企业部署受限于三个挑战:专有数据访问受限、权限管控环境下的危险操作执行,以及封闭系统无法从失败中学习。我们提出AOI(自主运维智能),一个可训练的多智能体框架,将自动化运维建模为受安全约束的结构化轨迹学习问题。该方法包含三个核心组件:首先,可训练诊断系统采用分组相对策略优化(GRPO),将专家知识提炼为本地部署的开源模型,实现基于偏好学习而无需暴露敏感数据;其次,读写分离执行架构将操作轨迹分解为观察、推理和执行阶段,确保安全学习的同时防止未经授权的状态修改;第三,失败轨迹闭环演化器挖掘失败轨迹并转化为修正性监督信号,实现持续数据增强。在AIOpsLab基准上评估显示,本研究贡献带来累积提升:(1) AOI运行时本身在全部86个任务上达到66.3% best@5成功率,优于先前最优水平(41.9%)24.4个百分点;(2) 加入观测器GRPO训练后,本地部署的140亿参数模型在63个未见故障类型的测试任务上达到42.9% avg@1,超越Claude Sonnet 4.5;(3) 演化器将37条失败轨迹转化为诊断指导,使端到端 avg@5 提升4.8个百分点,方差降低35%。
原文摘要 · Abstract (English)
Large language model (LLM) agents offer a promising data-driven approach to automating Site Reliability Engineering (SRE), yet their enterprise deployment is constrained by three challenges: restricted access to proprietary data, unsafe action execution under permission-governed environments, and the inability of closed systems to improve from failures. We present AOI (Autonomous Operations Intelligence), a trainable multi-agent framework formulating automated operations as a structured trajectory learning problem under security constraints. Our approach integrates three key components. First, a trainable diagnostic system applies Group Relative Policy Optimization (GRPO) to distill expert-level knowledge into locally deployed open-source models, enabling preference-based learning without exposing sensitive data. Second, a read-write separated execution architecture decomposes operational trajectories into observation, reasoning, and action phases, allowing safe learning while preventing unauthorized state mutation. Third, a Failure Trajectory Closed-Loop Evolver mines unsuccessful trajectories and converts them into corrective supervision signals, enabling continual data augmentation. Evaluated on the AIOpsLab benchmark, our contributions yield cumulative gains. (1) The AOI runtime alone achieves 66.3% best@5 success on all 86 tasks, outperforming the prior state-of-the-art (41.9%) by 24.4 points. (2) Adding Observer GRPO training, a locally deployed 14B model reaches 42.9% avg@1 on 63 held-out tasks with unseen fault types, surpassing Claude Sonnet 4.5. (3) The Evolver converts 37 failed trajectories into diagnostic guidance, improving end-to-end avg@5 by 4.8 points while reducing variance by 35%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。