用主动域适应提升日志异常检测，仅需2%人工标注即达93%准确率。
LogAction: Consistent Cross-system Anomaly Detection through Logs via Active Domain Adaptation
- 结合迁移学习与主动学习，利用成熟系统数据缓解冷启动问题。
- 在6组数据集上以2%标签实现平均93.01% F1分数，优于顶尖方法26.28%。
- 适合日志分析、系统监控等场景，尤其适用于标注成本高的环境。
基于日志的异常检测对保障软件系统可靠性与性能至关重要。然而，现有方法严重依赖标注数据，而大规模日志标注极具挑战。尽管已有不少基于迁移学习和主动学习的方法被提出，但其效果受限于源系统与目标系统间的数据分布差异及冷启动问题。本文提出LogAction，一种基于主动域适应的日志异常检测模型。该模型融合迁移学习与主动学习：一方面，利用成熟系统的标注数据训练基础模型，缓解主动学习中的冷启动问题；另一方面，采用基于自由能与不确定性的采样策略，选择分布在数据边界处的日志进行人工标注，以最小化标注代价有效缩小领域差距。在六组不同数据集组合上的实验表明，LogAction仅需2%的人工标注即可达到平均93.01%的F1分数，显著优于部分先进方法（提升26.28%）。
原文摘要 · Abstract (English)
Log-based anomaly detection is a essential task for ensuring the reliability and performance of software systems. However, the performance of existing anomaly detection methods heavily relies on labeling, while labeling a large volume of logs is highly challenging. To address this issue, many approaches based on transfer learning and active learning have been proposed. Nevertheless, their effectiveness is hindered by issues such as the gap between source and target system data distributions and cold-start problems. In this paper, we propose LogAction, a novel log-based anomaly detection model based on active domain adaptation. LogAction integrates transfer learning and active learning techniques. On one hand, it uses labeled data from a mature system to train a base model, mitigating the cold-start issue in active learning. On the other hand, LogAction utilize free energy-based sampling and uncertainty-based sampling to select logs located at the distribution boundaries for manual labeling, thus addresses the data distribution gap in transfer learning with minimal human labeling efforts. Experimental results on six different combinations of datasets demonstrate that LogAction achieves an average 93.01% F1 score with only 2% of manual labels, outperforming some state-of-the-art methods by 26.28%. Website: https://logaction.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。