用AI自动执行复杂的故障排查指南,提升IT系统故障诊断效率。
StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis
- 分三阶段自动化:质量优化、结构化解析、并行执行调度
- 在真实场景中达94%成功率,平行任务执行提速超32%
- 适合SRE工程师和需要高效运维的大型系统团队
大规模IT系统中的有效故障管理依赖于故障排查指南(TSG),但手动执行效率低且易出错。尽管大模型有望自动化此类任务,现有方案仍难以应对指南质量差、控制流复杂、数据查询密集及执行并行性不足等挑战。我们对92个真实TSG进行了实证研究,并据此提出StepFly——一个端到端的智能体框架。第一阶段通过工具TSG Mentor协助SRE提升指南质量;第二阶段利用大模型离线提取无结构指南中的有向无环图(DAG)并生成专用查询预处理插件(QPP);第三阶段在线执行时采用基于DAG的调度器-执行器架构与内存系统,确保流程正确并支持独立步骤并行。在真实TSG与事件上的评估显示,StepFly在GPT-4.1上实现约94%的成功率,优于基线且耗时与调用成本更低;对可并行化指南,执行时间减少32.9%至70.4%。代码与示例数据已公开于https://github.com/microsoft/StepFly。
原文摘要 · Abstract (English)
Effective incident management in large-scale IT systems relies on troubleshooting guides (TSGs), but their manual execution is slow and error-prone. While recent advances in LLMs offer promise for automating incident management tasks, existing LLM-based solutions lack specialized support for several key challenges, including managing TSG quality issues, interpreting complex control flow, handling data-intensive queries, and exploiting execution parallelism. We first conducted an empirical study on 92 real-world TSGs, and, guided by our findings, we present StepFly, a novel end-to-end agentic framework for troubleshooting guide automation. Our approach features a three-stage workflow: the first stage provides a comprehensive guide together with a tool, TSG Mentor, to assist site reliability engineers (SREs) in improving TSG quality; the second stage performs offline preprocessing using LLMs to extract structured execution directed acyclic graphs (DAGs) from unstructured TSGs and to create dedicated Query Preparation Plugins (QPPs); and the third stage executes online using a DAG-guided scheduler-executor framework with a memory system to ensure correct workflow and support parallel execution of independent steps. Our empirical evaluation on a collection of real-world TSGs and incidents demonstrates that StepFly achieves a ~94% success rate on GPT-4.1, outperforming baselines with less time and token consumption. Furthermore, it achieves a remarkable execution time reduction of 32.9% to 70.4% for parallelizable TSGs. Our code and sample data are publicly available at https://github.com/microsoft/StepFly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。