用自然语言生成日志查询，让开发者轻松应对多平台日志格式差异。
Chatting with Logs: An exploratory study on Finetuning LLMs for LogQL
- 构建自然语言到日志查询的转换接口，支持跨平台日志查询生成。
- 在真实数据集上微调后，模型生成准确查询能力提升75%。
- 适合需要处理多源日志的开发与运维人员使用。
现代分布式应用中，日志记录至关重要，但日志查询语言和格式缺乏标准化，导致开发者需掌握特定平台的查询语法及应用日志细节，负担过重。尽管大语言模型（LLMs）可用于生成日志查询，但现有模型因缺乏领域知识而表现不佳。为此，本文提出一种新的自然语言（NL）接口，使开发者仅通过自然语言输入即可生成目标日志查询语言的查询。我们进一步构建了~\textbf{NL2QL} 数据集，包含三类真实日志格式下的自然语言问题与对应 LogQL 查询，推动该领域的训练与评估。基于该数据集，我们对多个先进 LLM 进行微调与评估，结果表明微调后模型在生成准确 LogQL 查询方面显著优于未微调模型，最高提升达75%。进一步消融实验验证了额外训练数据的作用及跨日志格式的迁移能力。
原文摘要 · Abstract (English)
Logging is a critical function in modern distributed applications, but the lack of standardization in log query languages and formats creates significant challenges. Developers currently must write ad hoc queries in platform-specific languages, requiring expertise in both the query language and application-specific log details -- an impractical expectation given the variety of platforms and volume of logs and applications. While generating these queries with large language models (LLMs) seems intuitive, we show that current LLMs struggle with log-specific query generation due to the lack of exposure to domain-specific knowledge. We propose a novel natural language (NL) interface to address these inconsistencies and aide log query generation, enabling developers to create queries in a target log query language by providing NL inputs. We further introduce ~\textbf{NL2QL}, a manually annotated, real-world dataset of natural language questions paired with corresponding LogQL queries spread across three log formats, to promote the training and evaluation of NL-to-loq query systems. Using NL2QL, we subsequently fine-tune and evaluate several state of the art LLMs, and demonstrate their improved capability to generate accurate LogQL queries. We perform further ablation studies to demonstrate the effect of additional training data, and the transferability across different log formats. In our experiments, we find up to 75\% improvement of finetuned models to generate LogQL queries compared to non finetuned models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。