arXiv:2412.14011cs.CL2024-12被引 5

用大模型自动分析教师激励话语,提升教学评估效率

Towards an optimised evaluation of teachers' discourse: The case of engaging messages

  • 训练大模型从课堂录音中识别并分类教师激励语
  • 识别准确率达84.31%,分类准确率达91.11%
  • 发现高年级和学年末激励语使用减少,可针对性干预

教师话语对学生成绩有显著影响,但人工编码耗时费力。本研究提出新方法优化教师话语评估,基于两年真实课堂录音数据训练两个大语言模型,识别并分类中学教师使用的激励性话语。结果显示,模型在识别和分类上的敏感度分别为84.31%和91.11%,特异度为97.69%和86.36%。第二项研究将模型应用于第三年课堂数据,发现教师多使用强调参与益处的话语(与成绩提升相关),约三分之一提及不参与的负面影响(如焦虑增加);激励语使用在高年级和学年末显著下降。结果表明可通过干预优化激励话语使用,从而提升教学质量与学生表现。

原文摘要 · Abstract (English)

Evaluating teachers' skills is crucial for enhancing education quality and student outcomes. Teacher discourse, significantly influencing student performance, is a key component. However, coding this discourse can be laborious. This study addresses this issue by introducing a new methodology for optimising the assessment of teacher discourse. The research consisted of two studies, both within the framework of engaging messages used by secondary education teachers. The first study involved training two large language models on real-world examples from audio-recorded lessons over two academic years to identify and classify the engaging messages from the lessons' transcripts. This resulted in sensitivities of 84.31% and 91.11%, and specificities of 97.69% and 86.36% in identification and classification, respectively. The second study applied these models to transcripts of audio-recorded lessons from a third academic year to examine the frequency and distribution of message types by educational level and moment of the academic year. Results showed teachers predominantly use messages emphasising engagement benefits, linked to improved outcomes, while one-third highlighted non-engagement disadvantages, associated with increased anxiety. The use of engaging messages declined in Grade 12 and towards the academic year's end. These findings suggest potential interventions to optimise engaging message use, enhancing teaching quality and student outcomes.

教育技术大模型话语分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。