对比了大模型在日志异常检测中的表现，发现零样本大模型也能高效识别异常。
LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics
- 用提示工程让大模型直接分析日志，无需标注数据训练
- 微调的Transformer模型准确率最高（F1 0.96-0.99）
- 零样本大模型适合标签稀缺的真实场景，部署成本低
系统日志异常检测对保障大规模软件系统的可靠性至关重要，但传统方法难以应对现代日志数据的异构性和动态变化。近年来，大语言模型（LLMs）为日志理解提供了新路径，但其与传统方法的系统性比较仍不充分。本文在四个公开数据集（HDFS、BGL、Thunderbird、Spirit）上全面评估了三类方法：（1）经典日志解析器（Drain、Spell、AEL）结合机器学习分类器；（2）微调的Transformer模型（BERT、RoBERTa）；（3）提示驱动的LLM方法（GPT-3.5、GPT-4、LLaMA-3）在零样本和少样本设置下的表现。实验表明，尽管微调的Transformer模型取得最高F1分数（0.96–0.99），但提示式大模型展现出显著的零样本能力（F1：0.82–0.91），无需任何标注训练数据，这对真实环境中标注异常稀缺的场景具有重要意义。我们进一步分析了各方法的成本-精度权衡、延迟特性及失效模式，为实践者根据准确性、延迟、成本和标签可用性选择合适方法提供了可操作指导。所有代码与实验配置均已公开，便于复现。
原文摘要 · Abstract (English)
System log anomaly detection is critical for maintaining the reliability of large-scale software systems, yet traditional methods struggle with the heterogeneous and evolving nature of modern log data. Recent advances in Large Language Models (LLMs) offer promising new approaches to log understanding, but a systematic comparison of LLM-based methods against established techniques remains lacking. In this paper, we present a comprehensive benchmark study evaluating both LLM-based and traditional approaches for log anomaly detection across four widely-used public datasets: HDFS, BGL, Thunderbird, and Spirit. We evaluate three categories of methods: (1) classical log parsers (Drain, Spell, AEL) combined with machine learning classifiers, (2) fine-tuned transformer models (BERT, RoBERTa), and (3) prompt-based LLM approaches (GPT-3.5, GPT-4, LLaMA-3) in zero-shot and few-shot settings. Our experiments reveal that while fine-tuned transformers achieve the highest F1-scores (0.96-0.99), prompt-based LLMs demonstrate remarkablezero-shot capabilities (F1: 0.82-0.91) without requiring any labeled training data -- a significant advantage for real-world deployment where labeled anomalies are scarce. We further analyze the cost-accuracy trade-offs, latency characteristics, and failure modes of each approach. Our findings provide actionable guidelines for practitioners choosing log anomaly detection methods based on their specific constraints regarding accuracy, latency, cost, and label availability. All code and experimental configurations are publicly available to facilitate reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。