用可解释模型自动识别学生团队对话中的机制性思维时刻。
Locating acts of mechanistic reasoning in student team conversations with mechanistic machine learning
- 基于个体与小组发言构建时变概率模型,判断机制性推理发生
- 引入归纳偏置后模型在新学生、新情境下泛化能力提升
- 适合教育研究者用于分析学习对话,也利于模型设计者优化可解释性
STEM教育研究者常需识别学生对话中机制性推理的片段以深入分析,但人工筛选大量团队对话耗时费力。本文提出一种可解释机器学习模型,通过分析个体发言及小组贡献,输出学生进行机制性推理的时间变化概率。我们引入特定归纳偏置,引导概率动态朝领域一致行为演化。实验对比有无该偏置的模型,在包含未见过的学生和新讨论情境的语料上评估表现。结果表明,加入归纳偏置显著提升模型泛化能力,验证了可解释性是模型内在特性而非事后附加。最后,我们为教育研究者提供使用建议,为机器学习研究者提出扩展方向。总体目标是推动开发既可理解又可控的机制可解释模型,服务于未来教育研究。
原文摘要 · Abstract (English)
STEM education researchers are often interested in identifying moments of students' mechanistic reasoning for deeper analysis, but have limited capacity to search through many team conversation transcripts to find segments with a high concentration of such reasoning. We offer a solution in the form of an interpretable machine learning model that outputs time-varying probabilities that individual students are engaging in acts of mechanistic reasoning, leveraging evidence from their own utterances as well as contributions from the rest of the group. Using the toolkit of intentionally-designed probabilistic models, we introduce a specific inductive bias that steers the probabilistic dynamics toward desired, domain-aligned behavior. Experiments compare trained models with and without the inductive bias components, investigating whether their presence improves the desired model behavior on transcripts involving never-before-seen students and a novel discussion context. Our results show that the inductive bias improves generalization -- supporting the claim that interpretability is built into the model for this task rather than imposed post hoc. We conclude with practical recommendations for STEM education researchers seeking to adopt the tool and for ML researchers aiming to extend the model's design. Overall, we hope this work encourages the development of mechanistically interpretable models that are understandable and controllable for both end users and model designers in STEM education research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。