Socratic提问比提示优化更能提升学生长期编程学习能力
Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming

- 用对话式提问引导学生思考,而非直接优化提示词
- 对话组学生后期理解力更强,更倾向深度思考型提问
- 适合希望培养自主学习能力的教学场景
尽管大语言模型(LLMs)可提供个性化学习支持,但其教育应用仍存争议。本研究考察了两种基于LLM的导师对学生提示策略、学习效果及后续使用的影响:一种是通过对话提问引导的苏格拉底式指导(SG),另一种是优化提示词的提示精炼(PR)。在研究生移动机器人课程中,66名学生参与为期六周的干预,随后52人进行三周的无约束项目。结果显示,虽然两组在引导阶段的任务表现和提示模式相似,但SG组学生在后续学习中获得更高认知收益,且更倾向于采用以理解为导向的提示策略——该策略与更高理解水平显著相关。尽管学员感知SG效率较低,但结果表明苏格拉底式指导有助于长期提升学生自主使用LLM的能力,对LLM导师设计具有重要意义。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) can provide personalized support in learning, several studies have raised concerns regarding their use in education. Importantly, learning depends on how students engage with LLMs. This study examined how two types of LLM-based tutors shape students' prompting practices, learning, and subsequent LLM-use: a Socratic-Guidance (SG) tutor, which structures interaction through dialogic questioning, and a Prompt-Refinement (PR) tutor that guides the formulation of effective prompts. We conducted a two-phase study in a graduate-level mobile robotics course: 66 students used either the SG or PR tutor during a 6-week intervention, followed by 52 students using an unconstrained LLM during a 3-week course project. Results show that while the SG- and PR tutors led to similar task performance and prompting patterns during guided use, they differ in learning outcomes and later LLM-use. SG-students, relative to PR-student, achieved higher learning gains in later sessions, and were more likely to adopt understanding-driven prompting strategies, which are predictive of higher understanding, when using an unconstrained LLM. Although learners perceived the SG tutor as less efficient, the findings suggest that Socratic guidance supports the development of students' capacity to learn with LLMs over time, highlighting its importance for LLM tutor design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。