用大模型+静态分析生成编程任务的下一步提示,提升学习体验。
One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks
- 分三步生成:定目标、写代码、配说明,逐步引导。
- 静态分析控制代码质量,避免冗余或错误。
- 适合在线编程教学,尤其对初学者有帮助。
学生在学习编程时常因缺乏个性化指导而受阻,尤其是在线上环境中。本文提出一种新系统,通过结合大语言模型(LLM)与静态分析技术,为编程任务生成文本和代码两类下一步提示。该方法采用链式思维提示策略,分三步进行:(1) 从当前解法生成待完成的子目标列表;(2) 生成实现下一子目标的代码,并通过静态分析控制其大小与质量;(3) 生成描述该操作的自然语言说明。系统以开源 JetBrains Academy 插件为基础进行改造,支持集成在IDE中。我们设计了评估标准并开展两轮专家验证,最终在两所大学共14名学生的课堂实验中测试效果。结果显示,文本与代码提示均有效提升了学生推进编程任务的能力。
原文摘要 · Abstract (English)
Students often struggle with solving programming problems when learning to code, especially when they have to do it online, with one of the most common disadvantages of working online being the lack of personalized help. This help can be provided as next-step hint generation, i.e., showing a student what specific small step they need to do next to get to the correct solution. There are many ways to generate such hints, with large language models (LLMs) being among the most actively studied right now. While LLMs constitute a promising technology for providing personalized help, combining them with other techniques, such as static analysis, can significantly improve the output quality. In this work, we utilize this idea and propose a novel system to provide both textual and code hints for programming tasks. The pipeline of the proposed approach uses a chain-of-thought prompting technique and consists of three distinct steps: (1) generating subgoals - a list of actions to proceed with the task from the current student's solution, (2) generating the code to achieve the next subgoal, and (3) generating the text to describe this needed action. During the second step, we apply static analysis to the generated code to control its size and quality. The tool is implemented as a modification to the open-source JetBrains Academy plugin, supporting students in their in-IDE courses. To evaluate our approach, we propose a list of criteria for all steps in our pipeline and conduct two rounds of expert validation. Finally, we evaluate the next-step hints in a classroom with 14 students from two universities. Our results show that both forms of the hints - textual and code - were helpful for the students, and the proposed system helped them to proceed with the coding tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。