用轻量微调让大模型高效识别日志异常，节省资源且效果佳。
Adapting Large Language Models for Parameter-Efficient Log Anomaly Detection
- 采用LoRA和ReFT两种轻量微调技术适配大模型
- 在4个数据集上实现高精度异常检测，跨数据集泛化能力强
- 适合关注低资源日志安全监测的研究者与工程师
日志异常检测（LAD）旨在识别日志数据中异常模式，对系统安全与状态评估至关重要。尽管大语言模型（LLMs）在多个领域表现卓越，但其在日志异常检测中的应用仍处于探索阶段。由于全量微调成本过高，本文探索参数高效微调技术（PEFT）在适配LLMs用于LAD中的潜力。我们深入研究了两种主流的PEFT方法——低秩适应（LoRA）与表示微调（ReFT），并应用于三种不同规模的主流LLM（RoBERTa、GPT-2、Llama-3），实现参数高效的日志异常检测。在四个公开日志数据集上开展全面实验，揭示了基于PEFT的LLM驱动LAD在有效性、稳定性、样本效率、对不稳定日志的鲁棒性及跨数据集泛化能力等方面的诸多关键洞察。代码已开源。
原文摘要 · Abstract (English)
Log Anomaly Detection (LAD) seeks to identify atypical patterns in log data that are crucial to assessing the security and condition of systems. Although Large Language Models (LLMs) have shown tremendous success in various fields, the use of LLMs in enabling the detection of log anomalies is largely unexplored. This work aims to fill this gap. Due to the prohibitive costs involved in fully fine-tuning LLMs, we explore the use of parameter-efficient fine-tuning techniques (PEFTs) for adapting LLMs to LAD. To have an in-depth exploration of the potential of LLM-driven LAD, we present a comprehensive investigation of leveraging two of the most popular PEFTs -- Low-Rank Adaptation (LoRA) and Representation Fine-tuning (ReFT) -- to tap into three prominent LLMs of varying size, including RoBERTa, GPT-2, and Llama-3, for parameter-efficient LAD. Comprehensive experiments on four public log datasets are performed to reveal important insights into effective LLM-driven LAD in several key perspectives, including the efficacy of these PEFT-based LLM-driven LAD methods, their stability, sample efficiency, robustness w.r.t. unstable logs, and cross-dataset generalization. Code is available at https://github.com/mala-lab/LogADReft.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。