arXiv:2501.18005cs.SEcs.LG2025-01被引 6

用变异生成的堆栈信息微调大模型,精准定位生产环境软件崩溃根源。

Fault Localization via Fine-tuning Large Language Models with Mutation Generated Stack Traces

  • 通过代码变异生成百万级模拟崩溃数据,扩充训练集以微调LLM。
  • 在SAP HANA上实现66.9%根因定位准确率,远超基线12.6%和10.6%。
  • 方法无需源码或测试失败,适合无调试信息的线上故障分析。

软件崩溃通常难以分析,传统故障定位需依赖测试失败或源码,但生产环境仅提供崩溃日志与堆栈信息。本文提出一种仅基于堆栈信息的故障定位新方法:通过代码变异器在HANA代码库中注入410万次模拟崩溃,生成64,369个崩溃样本,微调大语言模型(LLM)。该方法可有效识别根因与最内层堆栈帧不一致的复杂案例。在SAP HANA上,定位准确率达66.9%,显著优于基线(12.6%和10.6%)。在SQLite和DuckDB上分别取得63%和74%准确率,验证了方法通用性。微调始终优于非微调提示方法。

原文摘要 · Abstract (English)

Abrupt and unexpected terminations of software are termed as software crashes. They can be challenging to analyze. Finding the root cause requires extensive manual effort and expertise to connect information sources like stack traces, source code, and logs. Typical approaches to fault localization require either test failures or source code. Crashes occurring in production environments, such as that of SAP HANA, provide solely crash logs and stack traces. We present a novel approach to localize faults based only on the stack trace information and no additional runtime information, by fine-tuning large language models (LLMs). We address complex cases where the root cause of a crash differs from the technical cause, and is not located in the innermost frame of the stack trace. As the number of historic crashes is insufficient to fine-tune LLMs, we augment our dataset by leveraging code mutators to inject synthetic crashes into the code base. By fine-tuning on 64,369 crashes resulting from 4.1 million mutations of the HANA code base, we can correctly predict the root cause location of a crash with an accuracy of 66.9\% while baselines only achieve 12.6% and 10.6%. We substantiate the generalizability of our approach by evaluating on two additional open-source databases, SQLite and DuckDB, achieving accuracies of 63% and 74%, respectively. Across all our experiments, fine-tuning consistently outperformed prompting non-finetuned LLMs for localizing faults in our datasets.

故障定位大模型代码变异堆栈分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。