用大模型提取语言学习反馈指标，效果接近真人教师。
Feedback Indicators: The Alignment between Llama and a Teacher in Language Learning
- 用Llama 3.1从学生作业中自动提取反馈指标
- 模型生成指标与真人评分高度相关，即使面对新组合也有效
- 为自动生成可解释的形成性反馈提供可靠基础
自动化反馈生成有望通过及时、精准的反馈提升学生学习进度，并帮助教师节省时间，聚焦更具战略性的教学工作。要生成高质量、信息丰富的形成性反馈，关键在于先提取相关反馈指标，这些指标是反馈构建的基础。教师常使用包含多种指标的反馈标准表，对学生的作答进行系统评估。本研究利用大语言模型Llama 3.1，从一门语言学习课程的学生提交内容中提取初始反馈指标，并分析了该模型生成的指标与人类评分在各类反馈标准下的对齐程度。结果显示，在多个反馈维度上均存在统计学意义的强相关性，甚至在未预期的指标-标准组合下依然成立。本文提出的方法为从学生作业中使用大模型提取反馈指标提供了有前景的路径，未来可用于自动生成可解释、透明的形成性反馈。
原文摘要 · Abstract (English)
Automated feedback generation has the potential to enhance students' learning progress by providing timely and targeted feedback. Moreover, it can assist teachers in optimizing their time, allowing them to focus on more strategic and personalized aspects of teaching. To generate high-quality, information-rich formative feedback, it is essential first to extract relevant indicators, as these serve as the foundation upon which the feedback is constructed. Teachers often employ feedback criteria grids composed of various indicators that they evaluate systematically. This study examines the initial phase of extracting such indicators from students' submissions of a language learning course using the large language model Llama 3.1. Accordingly, the alignment between indicators generated by the LLM and human ratings across various feedback criteria is investigated. The findings demonstrate statistically significant strong correlations, even in cases involving unanticipated combinations of indicators and criteria. The methodology employed in this paper offers a promising foundation for extracting indicators from students' submissions using LLMs. Such indicators can potentially be utilized to auto-generate explainable and transparent formative feedback in future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。