arXiv:2501.11031cs.SEcs.AI2025-01被引 20

用大小模型协作分析日志,既快又准还省钱。

AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model

  • 大小模型分工:小模型处理简单日志,大模型只在不确定时才调用。
  • 通过小模型的不确定性判断,减少大模型调用次数,节省成本。
  • 大模型通过参考历史错误案例提升推理能力,适合运维和系统工程师。

自动化日志分析对保障复杂系统的高可用性和可靠性至关重要。近年来,基于大语言模型(LLM)的自动日志分析引发广泛关注。当前主要分为两类:小语言模型(SLM)采用预训练+微调范式,在标注数据上微调以完成特定任务;而大语言模型则采用上下文学习范式,通过在提示中提供少量示例即可分析日志,无需参数更新。尽管各有优势,但SLM成本低但能力有限,而大模型虽强大却昂贵且效率低。为此,本文提出AdaptiveLog框架,通过协同使用大模型与小模型,在保证高性能的同时显著降低推理开销。该框架根据小模型的不确定性估计动态决定是否调用大模型,仅在小模型置信度低时启用大模型。此外,为增强大模型在日志分析中的推理能力,提出一种新提示策略:从历史数据中检索类似错误案例作为参考,使大模型能借鉴过往经验。大量实验表明,AdaptiveLog在多个任务上达到领先性能,显著提升日志分析准确率并保持高效成本控制。

原文摘要 · Abstract (English)

Automated log analysis is crucial to ensure high availability and reliability of complex systems. The advent of LLMs in NLP has ushered in a new era of language model-driven automated log analysis, garnering significant interest. Within this field, two primary paradigms based on language models for log analysis have become prominent. Small Language Models (SLMs) follow the pre-train and fine-tune paradigm, focusing on the specific log analysis task through fine-tuning on supervised datasets. On the other hand, LLMs following the in-context learning paradigm, analyze logs by providing a few examples in prompt contexts without updating parameters. Despite their respective strengths, we notice that SLMs are more cost-effective but less powerful, whereas LLMs with large parameters are highly powerful but expensive and inefficient. To trade-off between the performance and inference costs of both models in automated log analysis, this paper introduces an adaptive log analysis framework known as AdaptiveLog, which effectively reduces the costs associated with LLM while ensuring superior results. This framework collaborates an LLM and a small language model, strategically allocating the LLM to tackle complex logs while delegating simpler logs to the SLM. Specifically, to efficiently query the LLM, we propose an adaptive selection strategy based on the uncertainty estimation of the SLM, where the LLM is invoked only when the SLM is uncertain. In addition, to enhance the reasoning ability of the LLM in log analysis tasks, we propose a novel prompt strategy by retrieving similar error-prone cases as the reference, enabling the model to leverage past error experiences and learn solutions from these cases. Extensive experiments demonstrate that AdaptiveLog achieves state-of-the-art results across different tasks, elevating the overall accuracy of log analysis while maintaining cost efficiency.

日志分析大模型小模型智能运维

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。