AI可生成与教师相当的实验反馈,但纠错能力仍有差距。
Towards Adaptive Feedback with AI: Comparing the Feedback Quality of LLMs and Teachers on Experimentation Protocols
- 用LLM生成实验反馈,对比人类教师和专家评分
- 总体质量无显著差异,但纠错能力明显弱于真人
- 适合教育科技研究者,或想用AI辅助教学的教师
有效反馈对科学探究学习至关重要。随着人工智能发展,大语言模型(LLMs)提供了即时、自适应反馈的新可能,但其教学有效性缺乏实践验证。本研究评估并比较了LLM代理、教师及科学教育专家对学生实验方案反馈的质量。四位在科学探究与科学教育领域有经验的盲评人,基于六个维度(反馈上行、反馈下行、反馈前瞻、建设性语气、语言清晰度、专业术语使用)采用五级量表进行评分。结果显示,LLM生成的反馈在整体质量上与教师和专家无显著差异,但在‘反馈下行’(即识别并解释学生作业中的错误)维度表现较弱。定性分析表明,LLM在上下文理解与具体错误表达方面存在局限。研究建议将LLM反馈与人类专家结合,以提升教育效率与精准度。
原文摘要 · Abstract (English)
Effective feedback is essential for fostering students' success in scientific inquiry. With advancements in artificial intelligence, large language models (LLMs) offer new possibilities for delivering instant and adaptive feedback. However, this feedback often lacks the pedagogical validation provided by real-world practitioners. To address this limitation, our study evaluates and compares the feedback quality of LLM agents with that of human teachers and science education experts on student-written experimentation protocols. Four blinded raters, all professionals in scientific inquiry and science education, evaluated the feedback texts generated by 1) the LLM agent, 2) the teachers and 3) the science education experts using a five-point Likert scale based on six criteria of effective feedback: Feed Up, Feed Back, Feed Forward, Constructive Tone, Linguistic Clarity, and Technical Terminology. Our results indicate that LLM-generated feedback shows no significant difference to that of teachers and experts in overall quality. However, the LLM agent's performance lags in the Feed Back dimension, which involves identifying and explaining errors within the student's work context. Qualitative analysis highlighted the LLM agent's limitations in contextual understanding and in the clear communication of specific errors. Our findings suggest that combining LLM-generated feedback with human expertise can enhance educational practices by leveraging the efficiency of LLMs and the nuanced understanding of educators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。