用预训练模型实现持续学习,提升缺陷预测的适应性与准确性。
CodeFlowLM: Incremental Just-In-Time Defect Prediction with Pretrained Language Models and Exploratory Insights into Defect Localization
- 通过持续微调应对概念漂移和类别不平衡问题
- 在跨项目场景下最高提升68%的G-Mean指标
- 首次系统分析大模型在缺陷定位中的表现与局限
本文提出CodeFlowLM,一种基于预训练语言模型(PLM)的增量式即时缺陷预测框架。与传统在线学习不同,该框架采用持续微调机制,有效应对概念漂移、类别不平衡和验证延迟问题,无需从头训练。我们在编码器仅用和编码器-解码器结构的PLM(如CodeT5+和UniXCoder)上评估了其在项目内和跨项目场景下的性能,对比基线BORB。结果表明,CodeFlowLM在多场景下可实现最高68%的G-Mean提升,验证了其在动态软件环境中的卓越适应性和鲁棒性。进一步拓展至即时缺陷定位(JIT-DL),我们对比了GPT-5、Claude Sonnet 4.5和Gemini 2.5 Pro等大模型与基于注意力机制的模型。结果显示,尽管大模型在召回率@20%和努力度@20%上表现相当且更稳定,但注意力模型在细粒度排序指标(Top-k, IFA)上仍具优势。定性误差分析发现,多数误报源于人类保守偏见、补丁提示上下文不足以及JIT-Defects4J数据集可能存在标注错误。误报比例较低。整体而言,CodeFlowLM显著推进了增量式即时缺陷预测的前沿,同时为大模型在缺陷定位中的应用提供了关键洞察。
原文摘要 · Abstract (English)
This work introduces CodeFlowLM, an incremental learning framework for Just-In-Time Software Defect Prediction (JIT-SDP) that leverages pre-trained language models (PLMs). Unlike traditional online learners, CodeFlowLM employs continual fine-tuning to address concept drift, class imbalance, and verification latency without retraining from scratch. We evaluated encoder-only and encoder-decoder PLMs (notably CodeT5+ and UniXCoder) in JIT-SDP scenarios within and between projects, comparing them with the incremental baseline BORB. The results show that CodeFlowLM achieves up to 68% G-Mean gains, confirming its superior adaptability and robustness in evolving software environments. We further extend the analysis to Just-in-Time Defect Localization (JIT-DL), benchmarking Large Language Models (LLMs) such as GPT-5, Claude Sonnet 4.5, and Gemini 2.5 Pro against attention-based models. GPT-5 delivers comparable performance for Recall@20% and Effort@20% with higher stability, although attention-based methods retain an advantage in fine-grained ranking metrics (Top-k, IFA). A qualitative error analysis reveals that most false positives arise from (1) human-like conservative bias, (2) insufficient contextual information in diff-based prompts, and (3) potential dataset mislabeling in JIT-Defects4J. These findings highlight both the promise and the current limitations of LLM reasoning in defect localization. False negatives occur in smaller proportions. Overall, CodeFlowLM significantly advances the state of the art in incremental JIT-SDP, demonstrating superior adaptability and robustness in evolving software environments. Furthermore, our exploratory analysis of LLMs in JIT-DL not only benchmarks their performance against established attention-based models but also provides critical insights into the current limitations of prompt-based defect reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。