用大模型生成特定类型的编程初学者反馈，让辅导更精准。
You're (Not) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?
- 设计提示词让大模型按分类生成特定反馈类型。
- 验证了大模型能生成多样且符合分类的反馈内容。
- 适合教育AI和编程教学工具开发者参考。
反馈是影响学习的关键因素，传统上依赖专家经验制定。随着大语言模型（LLMs）兴起，自动化生成丰富、个性化的编程反馈成为可能。本文旨在利用LLMs为编程入门任务生成特定类型的反馈，重新审视现有反馈分类体系，以捕捉反馈的随机性、不确定性及变化程度等特征。通过迭代设计提示词，针对真实学生程序生成反馈，并评估其是否符合特定类型。结果表明，大模型能有效生成具有不同维度特征的反馈，深化了对反馈多样性的理解，为未来研究反馈效果与学习者信息需求提供了依据，并为面向新手程序员的AI辅助教学系统开发奠定了基础。
原文摘要 · Abstract (English)
Background: Feedback as one of the most influential factors for learning has been subject to a great body of research. It plays a key role in the development of educational technology systems and is traditionally rooted in deterministic feedback defined by experts and their experience. However, with the rise of generative AI and especially Large Language Models (LLMs), we expect feedback as part of learning systems to transform, especially for the context of programming. In the past, it was challenging to automate feedback for learners of programming. LLMs may create new possibilities to provide richer, and more individual feedback than ever before. Objectives: This paper aims to generate specific types of feedback for introductory programming tasks using LLMs. We revisit existing feedback taxonomies to capture the specifics of the generated feedback, such as randomness, uncertainty, and degrees of variation. Methods: We iteratively designed prompts for the generation of specific feedback types (as part of existing feedback taxonomies) in response to authentic student programs. We then evaluated the generated output and determined to what extent it reflected certain feedback types. Results and Conclusion: The present work provides a better understanding of different feedback dimensions and characteristics. The results have implications for future feedback research with regard to, for example, feedback effects and learners' informational needs. It further provides a basis for the development of new tools and learning systems for novice programmers including feedback generated by AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。