用机器学习分析大学生物理概念作答，自动判断其机制理解程度。
Combining physics education and machine learning research to measure evidence of students' mechanistic sensemaking
- 基于物理教育研究编码体系，用语言模型识别学生作答中的机制理解
- 工具与人工评判一致性强，不同模型在准确率和算力间有取舍
- 适合关注学生深层理解的教育研究者，助力自动化测评
机器学习的发展为科学教育研究提供了新可能。本文介绍了一种基于机器学习的工具设计进展，用于分析大学生在回答简短概念问题时展现的机制性理解。该工具采用与物理教育研究（PER）现有编码体系对齐的标注方案，并适配近期开发的语言编码器分类方法。我们对三种使用不同语言编码器的工具版本进行了试点测试，分析了学生书面回答中体现的理解水平。结果表明：第一，该工具在测量机制理解方面可与人工评判达成有效一致性；第二，编码器设计在准确率与计算开销之间存在权衡。文章讨论了该方法的潜力与局限，并举例说明未来如何支持物理教育研究。最后，作者反思了机器学习在支持教育研究中的作用，对教育研究者与机器学习研究者协同设计持谨慎乐观态度。
原文摘要 · Abstract (English)
Advances in machine learning (ML) offer new possibilities for science education research. We report on early progress in the design of an ML-based tool to analyze students' mechanistic sensemaking, working from a coding scheme that is aligned with previous work in physics education research (PER) and amenable to recently developed ML classification strategies using language encoders. We describe pilot tests of the tool, in three versions with different language encoders, to analyze sensemaking evident in college students' written responses to brief conceptual questions. The results show, first, that the tool's measurements of sensemaking can achieve useful agreement with a human coder, and, second, that encoder design choices entail a tradeoff between accuracy and computational expense. We discuss the promise and limitations of this approach, providing examples as to how this measurement scheme may serve PER in the future. We conclude with reflections on the use of ML to support PER research, with cautious optimism for strategies of co-design between PER and ML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。