用可解释的领域知识让大模型更好理解日志，准确率提升12%。
Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge
- 用带日志知识的自然文本持续预训练大模型，减少语言差异
- 在4个日志任务中平均准确率提升12.01%，优于现有方法
- 适合需要高精度日志分析的运维与故障诊断场景
日志分析是人工智能应用中的关键子领域，有助于实现大规模软件系统的自动故障与错误管理，替代传统人工方式。现有基于大语言模型（LLM）的方法受限于自然语言与日志语言之间的显著领域差距（后者包含大量如状态码、IP地址、资源路径等特定领域标记），影响实际应用效果。直接使用原始日志微调通用大模型会因标记分布不一致导致性能下降。本文提出一种领域适应方法，通过持续预训练（CPT）将可解释的领域知识融入开源大模型，以自然文本形式注入日志知识（而非原始日志），从而缓解分布偏差。为此，我们构建了NLPLog数据集，包含超过25万条日志相关问答对。基于此训练出的SuperLog模型在四个日志分析任务中表现最优，平均准确率较次优模型提升12.01%。消融实验进一步验证了使用可解释日志知识优于直接使用原始日志的优势。
原文摘要 · Abstract (English)
Log analysis represents a critical sub-domain within AI applications that facilitates automatic approaches to fault and error management of large-scaled software systems, saving labors of traditional manual methods. While existing solutions using large language models (LLMs) show promise, they are limited by a significant domain gap between natural and log languages (the latter contains rich domain-specific tokens such as status codes, IP addresses, resource pathes), which restricts their effectiveness in real-world applications. However, directly adapting general-purpose LLMs to log analysis using raw logs may degrade their performance due to inconsistent token distribution. In this paper, we present a domain adaptation approach that addresses these limitations by integrating interpretable domain knowledge into open-source LLMs through continual pre-training (CPT), which bridges this domain gap by adapting LLMs on interpretable natural texts with log knowledge (instead of raw logs) to reduce distribution discrepancy. To achieve this, we developed NLPLog, a comprehensive dataset containing over 250,000 question-answer pairs on log-related knowledge. Our resulting model, SuperLog, achieves the best performance across four log analysis tasks, with an average accuracy improvement of 12.01% over the second-best model. Ablation study also suggests advantages of domain adaption using interpretable log knowledge over using raw logs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。