arXiv:2506.17900cs.AIcs.DC2025-06中稿 · 2025 8th Internati…被引 4

用大模型自动分析日志,精准定位云上AI系统故障

Leveraging Large Language Model for Intelligent Log Processing and Autonomous Debugging in Cloud AI Platforms

  • 基于大模型构建多阶段语义推理框架,动态解析日志事件
  • 故障定位准确率提升16.2%,优于主流方法
  • 适合云平台运维与自动化调试场景

随着云平台AI系统复杂度和规模的快速增长,运行过程中产生的日志数据量巨大、非结构化且语义模糊,给故障定位与系统自修复带来巨大挑战。为此,本文提出一种基于大语言模型(LLM)的智能日志处理与自动调试框架——Intelligent Debugger (LLM-ID)。该方法在预训练Transformer基础上扩展,引入多阶段语义推理机制,实现对系统日志的上下文理解与故障链自动重构。首先,通过无监督聚类与嵌入机制动态结构化日志,提取事件模板与语义模式;其次,利用微调后的LLM结合多轮注意力机制对日志序列进行上下文推理,生成潜在故障假设与根因路径;进一步,引入基于强化学习的策略引导恢复规划器,根据LLM生成的修复策略支持云环境中的动态决策与自适应调试。实验结果表明,在云平台日志数据集上,LLM-ID相比现有规则引擎或传统日志分析系统,故障定位准确率提升16.2%,展现出更强的语义理解能力、持续学习能力与异构环境适应性。

原文摘要 · Abstract (English)

With the increasing complexity and rapid expansion of the scale of AI systems in cloud platforms, the log data generated during system operation is massive, unstructured, and semantically ambiguous, which brings great challenges to fault location and system self-repair. In order to solve this problem, this paper proposes an intelligent log processing and automatic debugging framework based on Large Language Model (LLM), named Intelligent Debugger (LLM-ID). This method is extended on the basis of the existing pre-trained Transformer model, and integrates a multi-stage semantic inference mechanism to realize the context understanding of system logs and the automatic reconstruction of fault chains. Firstly, the system log is dynamically structured, and the unsupervised clustering and embedding mechanism is used to extract the event template and semantic schema. Subsequently, the fine-tuned LLM combined with the multi-round attention mechanism to perform contextual reasoning on the log sequence to generate potential fault assumptions and root cause paths. Furthermore, this paper introduces a reinforcement learning-based policy-guided recovery planner, which is driven by the remediation strategy generated by LLM to support dynamic decision-making and adaptive debugging in the cloud environment. Compared with the existing rule engine or traditional log analysis system, the proposed model has stronger semantic understanding ability, continuous learning ability and heterogeneous environment adaptability. Experiments on the cloud platform log dataset show that LLM-ID improves the fault location accuracy by 16.2%, which is significantly better than the current mainstream methods

大模型日志分析故障定位自动化调试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。