用大模型即时评估学生自解释,让编程例题学习变主动。
Self-Explanation Tutor for Active Study of CS1 Worked Examples
- 用大模型实时分析学生对代码例题的自解释
- 学生修改次数增多,解释更完整且概念更深入
- 适合初学编程者提升主动学习能力
工作示例是入门编程教学的重要组成部分,但单纯阅读专家解析属于被动学习。自我解释——即学生以子目标层级分析问题与解法并自行阐述——能将学习转为主动任务,但难以规模化,因自由文本解释的自动评估与及时反馈缺乏有效方案。本文研究大语言模型(LLM)是否可填补这一空白。我们构建了面向入门级编程的自解释导师系统ESSE,学生需解释工作示例中的每行代码,系统即时提供基于LLM的正确性与完整性反馈。研究分两部分:第一,通过领域专家和非专家众包评分两种独立标准,评估LLM判断的可靠性及系统性偏差;第二,将系统部署于一门初级Java课程,发现使用该系统的学生更愿意持续修改而非放弃某行代码,其解释在多次尝试中逐渐更完整、更具概念深度,且表现出学习证据。结果表明,基于LLM的评估足以支撑自解释导师系统,且该系统能有效改善学生的学习行为。
原文摘要 · Abstract (English)
Worked examples are a important part of introductory programming, but reading their expert explanations is passive. Self explanation, students explaining the problem and its solution to themselves with subgoal level analysis, turns that study into an active task, yet it is hard to scale because assessing free-text explanations and returning timely feedback has had no easy automated solution. We investigate whether a large language model (LLM) can fill that gap. We build a self-explanation tutor for introductory programming, ESSE, in which students explain lines of worked examples and receive immediate LLM feedback on the correctness and completeness of each explanation, and we pursue two goals. First, we ask whether the LLM judges student explanations well enough to serve as the engine of the tutor; we assess its judgments against two independent human reference standards of different kinds, a single domain expert and a crowd of non-expert raters, each with its own strengths and weaknesses, characterizing both where the LLM is reliable and the systematic tendencies in how it diverges. Second, we ask whether the LLM-based tutoring benefits students; deploying it in an introductory Java course, we find that its feedback leads students to persist and revise rather than abandon a line, that their explanations grow more complete and conceptually richer across attempts, and that students show evidence of learning. These indicate that LLM-based assessment is good enough to power a self-explanation tutor, and that the tutor positively shapes how students study worked examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。