arXiv:2411.14303cs.SEcs.AI2024-11被引 24

用大模型自动生成编程调试练习题,提升学生找错能力。

BugSpotter: Automated Generation of Code Debugging Exercises

  • 用大模型根据题目描述生成带错误的代码,通过测试用例验证。
  • 自动生成题目难度适中,学生表现与人工题目相当。
  • 适合编程教学中快速构建调试训练任务,尤其对新手友好。

调试是编程学习中的关键技能,但不同课程重视程度不一。在代码生成大模型时代,学生需具备代码推理与错误识别能力,但常依赖试错而缺乏理解。本文提出 BugSpotter,利用大模型根据问题描述生成带缺陷的代码,并通过测试套件验证其正确性。学生通过设计使程序输出偏离预期的失败测试用例来互动,不仅锻炼调试能力,也强化对问题描述的理解。我们在大规模课堂中部署该工具,对比其生成的练习题与教师手工设计的题目发现:生成题目的难度分布合理,且与问题规范匹配良好;学生在两类题目上的表现无显著差异,表明 BugSpotter 能高效、有效支持调试教学。

原文摘要 · Abstract (English)

Debugging is an essential skill when learning to program, yet its instruction and emphasis often vary widely across introductory courses. In the era of code-generating large language models (LLMs), the ability for students to reason about code and identify errors is increasingly important. However, students frequently resort to trial-and-error methods to resolve bugs without fully understanding the underlying issues. Developing the ability to identify and hypothesize the cause of bugs is crucial but can be time-consuming to teach effectively through traditional means. This paper introduces BugSpotter, an innovative tool that leverages an LLM to generate buggy code from a problem description and verify the synthesized bugs via a test suite. Students interact with BugSpotter by designing failing test cases, where the buggy code's output differs from the expected result as defined by the problem specification. This not only provides opportunities for students to enhance their debugging skills, but also to practice reading and understanding problem specifications. We deployed BugSpotter in a large classroom setting and compared the debugging exercises it generated to exercises hand-crafted by an instructor for the same problems. We found that the LLM-generated exercises produced by BugSpotter varied in difficulty and were well-matched to the problem specifications. Importantly, the LLM-generated exercises were comparable to those manually created by instructors with respect to student performance, suggesting that BugSpotter could be an effective and efficient aid for learning debugging.

编程教育调试大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。