arXiv:2502.07786cs.SEcs.AI2025-02AAAI被引 12

用零样本学习和反例引导修复程序,让大模型更精准地修正初学者代码。

Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization

  • 结合MaxSAT定位错误位置,用程序草图引导大模型补全代码
  • 在6个大模型上提升修复率,修复量减少30%以上
  • 适合编程教学反馈,帮助学生理解错误而非盲目改写

针对编程课程中大量学生作业的自动化修复需求,现有方法存在局限:基于形式化方法的工具能准确定位错误但仅适用于控制流一致的程序;大语言模型虽能生成代码但常进行过度修改,不利于学习。本文提出一种新方法,利用MaxSAT进行故障定位,生成不含错误语句的程序草图,并通过零样本学习让大模型补全缺失部分。该过程遵循反例引导的CEGIS循环:若修复后程序不通过测试,则将反例反馈给模型继续优化。实验表明,该方法显著提升6种评估大模型的修复能力,可修复更多程序且修改量更小,优于其他配置及现有符号修复工具。

原文摘要 · Abstract (English)

Automated Program Repair (APR) for introductory programming assignments (IPAs) is motivated by the large number of student enrollments in programming courses each year. Since providing feedback on IPAs requires substantial time and effort from faculty, personalized feedback often involves suggesting fixes to students' programs. Formal Methods (FM)-based semantic repair approaches, check a program's execution against a test suite or reference solution, are effective but limited. These tools excel at identifying buggy parts but can only fix programs if the correct implementation and the faulty one share the same control flow graph. Conversely, Large Language Models (LLMs) are used for APR but often make extensive instead of minimal rewrites. This leads to more invasive fixes, making it harder for students to learn from their mistakes. In summary, LLMs excel at completing strings, while FM-based fault localization excel at identifying buggy parts of a program. In this paper, we propose a novel approach that combines the strengths of both FM-based fault localization and LLMs, via zero-shot learning, to enhance APR for IPAs. Our method uses MaxSAT-based fault localization to identify buggy parts of a program, then presents the LLM with a program sketch devoid of these buggy statements. This hybrid approach follows a CEGIS loop to iteratively refine the program. We ask the LLM to synthesize the missing parts, which are then checked against a test suite. If the suggested program is incorrect, a counterexample from the test suite is fed back to the LLM. Our experiments show that our counterexample guided approach, using MaxSAT-based bug-free program sketches, significantly improves the repair capabilities of all six evaluated LLMs. This method allows LLMs to repair more programs with smaller fixes, outperforming other configurations and state-of-the-art symbolic program repair tools.

程序修复大模型教育技术形式化方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。