用树结构+双阶段协同,让日志解析又快又准
SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration
- 构建双向树结构,正反向匹配提升模板识别率
- 先用轻量语法模型处理,仅在不确定时调用大模型
- 减少大模型调用次数,兼顾速度与准确率
日志解析是复杂系统自动化分析的关键步骤。传统基于启发式的方法效率高但易忽略语义上下文,准确性受限;而近期基于大模型的解析器虽能理解语义,却因频繁调用模型导致延迟过高。为此,我们提出SCOPE,首个自修正的在线日志解析方法,融合启发式与大模型的优势。SCOPE引入一种新型双向树结构,支持从前向和反向两个方向高效匹配模板,显著提升整体匹配率。同时采用两阶段句法-语义协同框架:首先通过轻量级NLP模型利用词性(POS)信息进行语法匹配,仅在存在不确定性时才调用大模型作为补充,大幅降低大模型接口调用频率,同时保持高精度。在多个基准数据集上的广泛评估表明,SCOPE在准确率与效率方面均优于现有最先进方法。代码与数据集已公开,以促进后续研究。
原文摘要 · Abstract (English)
Log parsing is a critical step for automated log analysis in complex systems. Traditional heuristic-based methods offer high efficiency but are limited in accuracy due to overlooking semantic context. In contrast, recent LLM-based parsers improve accuracy via se mantic understanding but incur high latency from frequent model calls. To address this, we propose SCOPE, the first self-correcting online log parsing method that integrates the strengths of both heuristic and LLM-based paradigms. SCOPE introduces a novel bi-directional tree structure that enables efficient template match ing from both forward and reverse directions, resulting in a higher overall matching rate. Additionally, it adopts a two-stage syntactic semantic collaboration framework: a lightweight NLP model first utilizes part-of-speech (POS) information for syntax-based match ing, while the LLM is selectively invoked as a fallback to handle semantically complex cases when uncertainty remains. This design significantly reduces LLM API usage while maintaining high ac curacy, achieving a balance between efficiency and effectiveness. Extensive evaluations on diverse benchmark datasets show that SCOPE outperforms state-of-the-art methods in both accuracy and efficiency. The implementation and datasets are publicly released to facilitate further research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。