用教学数据微调大模型,让编程辅导更启发学生而非直接给答案。
Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education
- 基于528组师生问答数据,用监督微调训练教学专用模型。
- 微调后模型提问引导多8%,用词精简58%,更符合教学逻辑。
- 适合教育科技开发者和想提升编程教学效率的教师参考。
大型语言模型(LLMs)在高等教育中日益受到关注,但其作为教学代理的有效性仍缺乏研究。本文提出GuideLM,一个专为编程教育设计的微调大模型,并集成至调试型C编译器DCC中,用于生成具有教学意义的错误解释。此前,DCC依赖通用OpenAI模型,虽准确但常过度协助学生,直接给出答案。为此,我们使用528组学生问题/教师回答对,对ChatGPT-4o和4o-mini进行监督微调,构建GuideLM与GuideLM-mini。通过专家分析每模型400条回复,基于建构主义与认知负荷理论评估概念支架、清晰度及苏格拉底式引导。结果表明,两模型在苏格拉底引导上提升8%,用词经济性提高58%,优于GPT-4o;但通用准确性略有下降。研究显示,针对教育场景的数据微调是提升模型教学适配性的有效路径。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly being explored in higher education, yet their effectiveness as teaching agents remains underexamined. In this paper, we present the development of GuideLM, a fine-tuned LLM designed for programming education. GuideLM has been integrated into the Debugging C Compiler (DCC), an educational C compiler that leverages LLMs to generate pedagogically sound error explanations. Previously, DCC relied on off-the-shelf OpenAI models, which, while accurate, often over-assisted students by directly providing solutions despite contrary prompting. To address this, we employed supervised fine-tuning (SFT) on a dataset of 528 student-question/teacher-answer pairs, creating two models: GuideLM and GuideLM-mini, fine-tuned on ChatGPT-4o and 4o-mini, respectively. We conducted an expert analysis of 400 responses per model, comparing their pedagogical effectiveness against base OpenAI models. Our evaluation, grounded in constructivism and cognitive load theory, assessed factors such as conceptual scaffolding, clarity, and Socratic guidance. Results indicate that GuideLM and GuideLM-mini improve pedagogical performance, with an 8% increase in Socratic guidance and a 58% improvement in economy of words compared to GPT-4o. However, this refinement comes at the cost of a slight reduction in general accuracy. While further work is needed, our findings suggest that fine-tuning LLMs with targeted datasets is a promising approach for developing models better suited to educational contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。