arXiv:2409.13561cs.SEcs.CL2024-09中稿 · the 35th IEEE Inte…被引 22

自动提取日志中的故障线索,提升系统故障诊断效率。

Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis

  • 通过语义相似性粗筛+提示词微调的预训练模型,精准定位故障日志。
  • 在真实工业数据上比最佳基线(ChatGPT)F1提升25.8~37.9个百分点。
  • 适用于运维工程师快速定位故障,降低人工排查成本。

日志在在线服务系统维护中至关重要,常包含有效故障缓解的关键信息。尽管现有异常检测方法能识别大量运行数据中的异常日志,但工程师仍需手动分析日志消息以理解故障,过程繁琐且易出错。通过对CloudA的日志排查实践分析发现,工程师通常重点关注两类日志信息:故障描述(记录异常系统事件)和故障参数(标识相关实体)。基于此,我们提出一种名为LoFI的自动化方法,从日志中提取故障指示信息。LoFI包含两个阶段:第一阶段基于语义相似性进行粗粒度筛选,收集与故障相关的日志;第二阶段利用预训练语言模型结合新型提示词微调方法,从筛选出的日志中提取细粒度关注信息。我们在Apache Spark及CloudA的工业数据集上评估了LoFI,结果表明其显著优于所有基线方法,相比最佳基线(ChatGPT)F1绝对提升25.8~37.9。此外,LoFI已在CloudA成功部署,并通过用户研究验证了其有效性。代码与数据已开源。

原文摘要 · Abstract (English)

Logs are imperative in the maintenance of online service systems, which often encompass important information for effective failure mitigation. While existing anomaly detection methodologies facilitate the identification of anomalous logs within extensive runtime data, manual investigation of log messages by engineers remains essential to comprehend faults, which is labor-intensive and error-prone. Upon examining the log-based troubleshooting practices at CloudA, we find that engineers typically prioritize two categories of log information for diagnosis. These include fault-indicating descriptions, which record abnormal system events, and fault-indicating parameters, which specify the associated entities. Motivated by this finding, we propose an approach to automatically extract such faultindicating information from logs for fault diagnosis, named LoFI. LoFI comprises two key stages. In the first stage, LoFI performs coarse-grained filtering to collect logs related to the faults based on semantic similarity. In the second stage, LoFI leverages a pre-trained language model with a novel prompt-based tuning method to extract fine-grained information of interest from the collected logs. We evaluate LoFI on logs collected from Apache Spark and an industrial dataset from CloudA. The experimental results demonstrate that LoFI outperforms all baseline methods by a significant margin, achieving an absolute improvement of 25.8~37.9 in F1 over the best baseline method, ChatGPT. This highlights the effectiveness of LoFI in recognizing fault-indicating information. Furthermore, the successful deployment of LoFI at CloudA and user studies validate the utility of our method. The code and data are available at https://github.com/Jun-jie-Huang/LoFI.

日志分析故障诊断大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。