测试大模型在高校教学中生成反馈的效果与局限。
Evaluation of Large Language Models' educational feedback in Higher Education: potential, limitations and implications for educational practice
- 用教师设计的评分标准让7个大模型生成反馈。
- 模型反馈结构良好,能有效支持形成性学习。
- 适合教育技术研究者和高校教师参考。
高等教育中反馈管理的重要性已得到广泛认可,其在提升教学、学习和评估过程方面起着关键作用。当前教育环境中,反馈实践正日益受到人工智能(AI)技术的影响。理解AI对反馈生成的影响,对于识别其潜在优势并制定有效实施策略至关重要。本研究采用成熟分析框架,评估大语言模型(LLMs)生成的反馈对学生学习的支持作用。具体而言,将7个不同LLM置于一个关于包容性教学与学习的培训课程项目中,基于由大学教师制定的结构化评分量表(包含特定标准与表现水平),要求模型生成定量评分与定性反馈。随后,利用Hughes、Smith与Creese的框架对生成的AI反馈进行分析,以评估其结构与促进形成性学习的有效性。总体结果显示,LLMs能够生成结构良好的反馈,具有作为可持续且有意义反馈工具的巨大潜力,前提是提供清晰的情境信息与明确指令,这将在结论部分进一步探讨。
原文摘要 · Abstract (English)
The importance of managing feedback practices in higher education has been widely recognised, as they play a crucial role in enhancing teaching, learning, and assessment processes. In today's educational landscape, feedback practices are increasingly influenced by technological advancements, particularly artificial intelligence (AI). Understanding the impact of AI on feedback generation is essential for identifying its potential benefits and establishing effective implementation strategies. This study examines how AI-generated feedback supports student learning using a well-established analytical framework. Specifically, feedback produced by different Large Language Models (LLMs) was assessed in relation to student-designed projects within a training course on inclusive teaching and learning. The evaluation process involved providing seven LLMs with a structured rubric, developed by the university instructor, which defined specific criteria and performance levels. The LLMs were tasked with generating both quantitative assessments and qualitative feedback based on this rubric. The AI-generated feedback was then analysed using Hughes, Smith, and Creese's framework to evaluate its structure and effectiveness in fostering formative learning experiences. Overall, these findings indicate that LLMs can generate well-structured feedback and hold great potential as a sustainable and meaningful feedback tool, provided they are guided by clear contextual information and a well-defined instructions that will be explored further in the conclusions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。