自动诊断系统快速定位复杂数据产品故障根源。
Automated Root Cause Analysis System for Complex Data Products
- 用领域语言构建可并行执行的自诊断指南
- 通过大模型智能排序输出，近实时完成修复
- 适合需快速响应的数据平台运维团队
我们提出ARCAS（Automated Root Cause Analysis System），一个基于领域特定语言（DSL）的诊断平台，支持快速实现诊断逻辑且学习成本低。ARCAS由一组可并行运行的自动化故障排查指南（Auto-TSGs）构成，能利用产品遥测数据检测问题并近实时实施缓解措施。该DSL专为领域专家设计，使其能在短时间内交付高度定制化的Auto-TSGs，无需理解其与平台其余部分的交互机制，从而显著缩短故障响应时间，节省关键工程资源。相比Datadog和New Relic等主要聚焦监控且需人工干预的平台，ARCAS通过大语言模型（LLM）对Auto-TSG输出进行优先级排序并自主采取行动，避免了对系统整体行为的复杂理解。我们阐述了ARCAS的核心理念，并展示了其在Azure Synapse Analytics和Microsoft Fabric Synapse Data Warehouse多个产品中的成功应用。
原文摘要 · Abstract (English)
We present ARCAS (Automated Root Cause Analysis System), a diagnostic platform based on a Domain Specific Language (DSL) built for fast diagnostic implementation and low learning curve. Arcas is composed of a constellation of automated troubleshooting guides (Auto-TSGs) that can execute in parallel to detect issues using product telemetry and apply mitigation in near-real-time. The DSL is tailored specifically to ensure that subject matter experts can deliver highly curated and relevant Auto-TSGs in a short time without having to understand how they will interact with the rest of the diagnostic platform, thus reducing time-to-mitigate and saving crucial engineering cycles when they matter most. This contrasts with platforms like Datadog and New Relic, which primarily focus on monitoring and require manual intervention for mitigation. ARCAS uses a Large Language Model (LLM) to prioritize Auto-TSGs outputs and take appropriate actions, thus suppressing the costly requirement of understanding the general behavior of the system. We explain the key concepts behind ARCAS and demonstrate how it has been successfully used for multiple products across Azure Synapse Analytics and Microsoft Fabric Synapse Data Warehouse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。