arXiv:2409.20503cs.SEcs.AI2024-09被引 1

探究日志中哪些信息对异常检测有效,发现事件出现频率最关键。

What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approach

  • 设计可配置的Transformer模型,融合语义、顺序和时间信息
  • 实验表明事件发生频次是检测异常的关键因素
  • 适合关注日志分析与系统监控的工程师和研究者

日志数据来自源代码中的打印语句,反映软件运行过程。现有基于深度学习的日志异常检测方法多关注语义或序列信息,但各类信息的作用尚不明确,且普遍忽略日志的时间戳。本文提出一种可配置的Transformer模型,能同时捕获日志的语义、顺序与时间信息,并支持灵活输入不同长度的日志序列,突破了传统方法对固定长度或时间窗口的限制。通过多种特征组合实验,结果表明:在所用公开数据集上,事件发生信息对异常检测起决定性作用;而顺序与时间信息影响较小。这反映出当前公开数据集较为简单,提示需构建包含多样化异常的新数据集以更全面评估模型性能。

原文摘要 · Abstract (English)

Log data are generated from logging statements in the source code, providing insights into the execution processes of software applications and systems. State-of-the-art log-based anomaly detection approaches typically leverage deep learning models to capture the semantic or sequential information in the log data and detect anomalous runtime behaviors. However, the impacts of these different types of information are not clear. In addition, most existing approaches ignore the timestamps in log data, which can potentially provide fine-grained sequential and temporal information. In this work, we propose a configurable Transformer-based anomaly detection model that can capture the semantic, sequential, and temporal information in the log data and allows us to configure the different types of information as the model's features. Additionally, we train and evaluate the proposed model using log sequences of different lengths, thus overcoming the constraint of existing methods that rely on fixed-length or time-windowed log sequences as inputs. With the proposed model, we conduct a series of experiments with different combinations of input features to evaluate the roles of different types of information in anomaly detection. The model can attain competitive and consistently stable performance compared to the baselines when presented with log sequences of varying lengths. The results indicate that the event occurrence information plays a key role in identifying anomalies, while the impact of the sequential and temporal information is not significant for anomaly detection on the studied public datasets. On the other hand, the findings also reveal the simplicity of the studied public datasets and highlight the importance of constructing new datasets that contain different types of anomalies to better evaluate the performance of anomaly detection models.

日志分析异常检测Transformer时序数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。