用大模型分析失败SQL，提升数据库回放故障根因定位准确率
On Enhancing Root Cause Analysis with SQL Summaries for Failures in Database Workload Replays at SAP HANA
- 引入大模型提取失败SQL摘要作为新特征
- F1-Macro得分提升4.77%
- 适合需要精准定位数据库回放故障的工程师
捕获数据库工作负载并重放至新版本数据库是有效的回归测试方法。然而，数据隐私限制、时间依赖性或多线程环境中的非确定性等因素可能导致大量误报，影响测试效果。为此，我们采用基于机器学习的框架自动化故障根因分析。但模型对训练数据外新问题的泛化能力不足仍是挑战。仅通过新增故障重新训练效果有限，因不同根因间特征存在重叠。我们利用大语言模型（LLM）分析失败的SQL语句，提取简洁的失败摘要作为额外特征，增强分类性能。实验表明，在我们的数据上F1-Macro分数提升了4.77%。该方法有助于为用户提供更多洞察，改善回放结果评估。
原文摘要 · Abstract (English)
Capturing the workload of a database and replaying this workload for a new version of the database can be an effective approach for regression testing. However, false positive errors caused by many factors such as data privacy limitations, time dependency or non-determinism in multi-threaded environment can negatively impact the effectiveness. Therefore, we employ a machine learning based framework to automate the root cause analysis of failures found during replays. However, handling unseen novel issues not found in the training data is one general challenge of machine learning approaches with respect to generalizability of the learned model. We describe how we continue to address this challenge for more robust long-term solutions. From our experience, retraining with new failures is inadequate due to features overlapping across distinct root causes. Hence, we leverage a large language model (LLM) to analyze failed SQL statements and extract concise failure summaries as an additional feature to enhance the classification process. Our experiments show the F1-Macro score improved by 4.77% for our data. We consider our approach beneficial for providing end users with additional information to gain more insights into the found issues and to improve the assessment of the replay results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。