用知识增强提升日志异常检测准确率,无需训练即可部署。
Log Anomaly Detection with Large Language Models via Knowledge-Enriched Fusion
- 通过检索增强生成融合上下文知识,增强原始日志语义。
- 在四个大规模数据集上优于五个基线方法,尤其擅长处理模糊日志。
- 无需训练、推理高效,适合实际系统部署使用。
系统日志是监控和管理分布式系统的关键资源,可提供故障和异常行为的洞察。传统日志分析方法(如基于模板和序列驱动的方法)常丢失重要语义信息或难以处理模糊日志模式。为此,我们提出EnrichLog——一种无需训练、基于条目的异常检测框架,通过融合语料特定和样本特定的知识来增强原始日志条目。EnrichLog利用检索增强生成技术整合相关上下文知识,无需重新训练即可实现更准确、可解释的异常检测。我们在四个大规模系统日志基准数据集上评估EnrichLog,对比五种基线方法。结果表明,EnrichLog在异常检测性能上持续领先,有效处理模糊日志条目,并保持高效推理。同时引入语料与样本级知识显著提升模型置信度与检测准确率,使其适用于实际部署。
原文摘要 · Abstract (English)
System logs are a critical resource for monitoring and managing distributed systems, providing insights into failures and anomalous behavior. Traditional log analysis techniques, including template-based and sequence-driven approaches, often lose important semantic information or struggle with ambiguous log patterns. To address this, we present EnrichLog, a training-free, entry-based anomaly detection framework that enriches raw log entries with both corpus-specific and sample-specific knowledge. EnrichLog incorporates contextual information, including historical examples and reasoning derived from the corpus, to enable more accurate and interpretable anomaly detection. The framework leverages retrieval-augmented generation to integrate relevant contextual knowledge without requiring retraining. We evaluate EnrichLog on four large-scale system log benchmark datasets and compare it against five baseline methods. Our results show that EnrichLog consistently improves anomaly detection performance, effectively handles ambiguous log entries, and maintains efficient inference. Furthermore, incorporating both corpus- and sample-specific knowledge enhances model confidence and detection accuracy, making EnrichLog well-suited for practical deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。