arXiv:2608.14065cs.SEcs.AI2026-08中稿 · ESEM 2026被引 1

LLM修代码效果受复杂度和定位精度影响,低成本模型也能高效修复中等复杂度漏洞。

Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency

论文配图:Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency
图 1 · 摘自论文原文
  • 用多维度实验对比不同LLM和修复策略在各类漏洞上的表现
  • 超过50%中等复杂度漏洞可被低成本模型修复,但定位不准会拉大性能差距
  • 高成本模型未必更优,GPT-5修复能力最强,但DeepSeek-V3.2性价比最高

软件漏洞仍是开发中的关键挑战,亟需高效的自动化程序修复(APR)技术。尽管基于大语言模型(LLM)的APR系统展现出潜力,但以往研究多关注整体修复效果,对漏洞复杂度、故障定位精度、推理设置及修复成本效益的影响仍缺乏深入探讨。本研究通过多维实证框架与统计分析,评估两种APR技术(ChatRepair与CodeCorrector)在三种LLM(DeepSeek、GPT、Llama)下的表现,覆盖不同复杂度与定位策略。结果表明:虽然结构复杂的漏洞和不精确的定位会增加修复难度,但基于LLM的APR仍具竞争力;定位不准确会显著扩大不同技术间的性能差距。此外,更高成本的LLM与更强推理设置并未持续提升成本效益,揭示了修复效果与计算开销间的非线性权衡。超过50%的中等复杂度漏洞可被低成本模型修复;GPT-5比DeepSeek-V4-pro和DeepSeek-V3.2分别多修复7个和39个更复杂漏洞;而DeepSeek-V3.2总修复成本最低,表现出最优成本效益。

原文摘要 · Abstract (English)

Background: Software bugs remain a critical challenge in development, necessitating effective Automated Program Repair (APR) techniques. While Large Language Model (LLM)-based APR systems have shown promise, prior studies primarily focus on overall repair effectiveness. The effects of bug complexity, fault localization, reasoning settings, and repair cost-effectiveness remain insufficiently explored. Aims: This study presents a comprehensive empirical analysis of LLM-based APR, focusing on how repair performance is shaped by bug complexity, fault localization, reasoning settings, and costs. Method: We evaluate two APR techniques (ChatRepair and CodeCorrector) using three LLMs (DeepSeek, GPT, and Llama), and examine their performance across diverse levels of bug complexity and localization strategies through a multi-dimensional empirical framework and statistical analysis. Results: Although structurally complex bugs and imprecise fault localization make repair more challenging, LLM-based APR techniques still achieve competitive repair effectiveness. Imprecise fault localization can substantially enlarge the performance gap between APR techniques. Furthermore, higher-cost LLMs and stronger reasoning settings do not consistently yield better cost-efficiency, revealing a nontrivial trade-off between repair effectiveness and computational cost. Conclusions: Over 50% of moderately complex bugs can be repaired by low-cost LLM-based APR techniques. The repair effectiveness gap between APR techniques becomes larger as fault localization becomes less precise. GPT-5 repairs 7 and 39 more complex bugs than DeepSeek-V4-pro and DeepSeek-V3.2, respectively; whereas the total repair cost of DeepSeek-V3.2 shows the best cost-efficiency performance.

程序修复大模型成本效率漏洞检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。