自动将日志正则表达式转为Dynatrace语言，减少迁移成本
Lost in Translation? Converting RegExes for Log Parsing into Dynatrace Pattern Language
- 结合规则匹配与GPT-4优化，实现正则到DPL的自动化转换
- 在946个真实正则中成功转换73.7%，优化后F1和MCC均超0.91
- 适合需迁移到Dynatrace等平台的企业用户
日志文件为检测和诊断企业软件应用及数据中心问题提供重要信息。已有多种日志分析工具使用正则表达式（RegEx）进行信息提取。近年来，商业平台如Groq和Dynatrace Pattern Language（DPL）引入了领域专用语言以提升日志解析效率。但用户迁移时需手动重写正则，耗时且易出错。本文提出Reptile，结合规则方法与最佳尝试策略实现从RegEx到DPL的转换，并集成GPT-4对生成的DPL模式进行优化。在某大型公司收集的946个正则上评估显示，Reptile安全转换率达73.7%；对23个真实正则的优化评估表明，其F1-score和马修相关系数（MCC）均高于0.91。结果表明该方法具有显著实用价值，可有效支持企业向现代日志分析平台（如Dynatrace）迁移。
原文摘要 · Abstract (English)
Log files provide valuable information for detecting and diagnosing problems in enterprise software applications and data centers. Several log analytics tools and platforms were developed to help filter and extract information from logs, typically using regular expressions (RegExes). Recent commercial log analytics platforms provide domain-specific languages specifically designed for log parsing, such as Grok or the Dynatrace Pattern Language (DPL). However, users who want to migrate to these platforms must manually convert their RegExes into the new pattern language, which is costly and error-prone. In this work, we present Reptile, which combines a rule-based approach for converting RegExes into DPL patterns with a best-effort approach for cases where a full conversion is impossible. Furthermore, it integrates GPT-4 to optimize the obtained DPL patterns. The evaluation with 946 RegExes collected from a large company shows that Reptile safely converted 73.7% of them. The evaluation of Reptile's pattern optimization with 23 real-world RegExes showed an F1-score and MCC above 0.91. These results are promising and have ample practical implications for companies that migrate to a modern log analytics platform, such as Dynatrace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。