KAT实现大模型分布式推理的纳米级异常检测与因果分析
Kunlun Anomaly Troubleshooter: Enabling Kernel-Level Anomaly Detection and Causal Reasoning for Large Model Distributed Inference
- 通过GPU工作者同步性，用函数追踪数据定位内核级异常
- 异常检测精度超0.884，召回率达0.936，显著缩小诊断范围
- 结合领域适配大模型，自动生成自然语言故障解释，适合运维人员
大模型分布式推理（LMDI）中的异常排查仍是一大挑战。性能下降或延迟抖动等问题需依赖领域专家手动诊断，过程耗时且准确率低。本文提出Kunlun异常排查框架（KAT），首次专为LMDI设计。其核心创新有二：首先，利用GPU工作节点的同步性，创新性地基于函数追踪数据，在纳秒级分辨率下精准检测内核级异常及关联硬件组件；其次，将检测结果融入领域适配的大语言模型，实现复杂异常症状的系统性因果推理与自然语言解释。在阿里云生产环境的评估显示，KAT在异常检测上达到0.884以上精度和0.936召回率，提供细粒度异常洞察，显著缩小诊断范围，提升排查效率与成功率。
原文摘要 · Abstract (English)
Anomaly troubleshooting for large model distributed inference (LMDI) remains a critical challenge. Resolving anomalies such as inference performance degradation or latency jitter in distributed system demands significant manual efforts from domain experts, resulting in extremely time-consuming diagnosis processes with relatively low accuracy. In this paper, we introduce Kunlun Anomaly Troubleshooter (KAT), the first anomaly troubleshooting framework tailored for LMDI. KAT addresses this problem through two core innovations. First, KAT exploits the synchronicity and consistency of GPU workers, innovatively leverages function trace data to precisely detect kernel-level anomalies and associated hardware components at nanosecond resolution. Second, KAT integrates these detection results into a domain-adapted LLM, delivering systematic causal reasoning and natural language interpretation of complex anomaly symptoms. Evaluations conducted in Alibaba Cloud Service production environment indicate that KAT achieves over 0.884 precision and 0.936 recall in anomaly detection, providing detail anomaly insights that significantly narrow down the diagnostic scope and improve both the efficiency and success rate of troubleshooting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。