用协同变压器统一检测系统日志中的单点与群体异常,准确率超99%。
A unified framework for detecting point and collective anomalies in operating system logs via collaborative transformers
- 通过协同变压器融合多模态日志信息,捕捉不同日志间的交互关系。
- 在7个基准数据集上平均精确率、召回率和F1均达99.6%以上。
- 适合网络安全监控与系统运维场景,对复杂日志分析有强适应性。
日志异常检测对保障操作系统安全至关重要。由于日志数据来源不同,记录的信息可视为多种日志模态。现有单模态方法常忽略模态差异,而多模态方法难以处理模态间交互。受多模态情感分析启发,本文提出CoLog框架,通过协同变压器与多头印象注意力机制,联合编码多种日志模态,学习其交互关系,实现全面异常检测。为应对模态异构性,CoLog引入模态自适应层,对不同模态表示进行适配。该方法能学习数据中的细微模式与依赖,显著提升检测能力。大量实验表明,CoLog优于现有最先进方法。在检测单点与群体异常时,其在7个基准数据集上的平均精确率为99.63%,平均召回率为99.59%,平均F1得分为99.61%。其综合检测能力使其在网络安全、系统监控和运营效率领域具有重要应用价值。CoLog代表了日志异常检测的重要进展,提供了一个统一且高效的解决方案。代码已开源:https://github.com/NasirzadehMoh/CoLog。
原文摘要 · Abstract (English)
Log anomaly detection is crucial for preserving the security of operating systems. Depending on the source of log data collection, various information is recorded in logs that can be considered log modalities. In light of this intuition, unimodal methods often struggle by ignoring the different modalities of log data. Meanwhile, multimodal methods fail to handle the interactions between these modalities. Applying multimodal sentiment analysis to log anomaly detection, we propose CoLog, a framework that collaboratively encodes logs utilizing various modalities. CoLog utilizes collaborative transformers and multi-head impressed attention to learn interactions among several modalities, ensuring comprehensive anomaly detection. To handle the heterogeneity caused by these interactions, CoLog incorporates a modality adaptation layer, which adapts the representations from different log modalities. This methodology enables CoLog to learn nuanced patterns and dependencies within the data, enhancing its anomaly detection capabilities. Extensive experiments demonstrate CoLog's superiority over existing state-of-the-art methods. Furthermore, in detecting both point and collective anomalies, CoLog achieves a mean precision of 99.63%, a mean recall of 99.59%, and a mean F1 score of 99.61% across seven benchmark datasets for log-based anomaly detection. The comprehensive detection capabilities of CoLog make it highly suitable for cybersecurity, system monitoring, and operational efficiency. CoLog represents a significant advancement in log anomaly detection, providing a sophisticated and effective solution to point and collective anomaly detection through a unified framework and a solution to the complex challenges automatic log data analysis poses. We also provide the implementation of CoLog at https://github.com/NasirzadehMoh/CoLog.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。