探究可解释的学情模型是否真能帮教师做教学决策。
Does Interpretability of Knowledge Tracing Models Support Teacher Decision Making?
- 用模拟和真人实验对比可解释与不可解释模型的教学效果。
- 教师虽更信任可解释模型,但实际教学效率无显著差异。
- 教师不完全依赖模型,需研究师生如何理解使用模型。
知识追踪(KT)模型是教学决策的重要依据,例如决定下一步该教什么任务或何时停止教授某一技能。由于教学决策影响重大,通常要求KT模型具备可解释性,即应有明确的人类学习机制,并提供对学习者能力的明确估计。然而,目前尚无研究验证这种可解释性是否真正帮助教师做出更好决策。本文首先通过模拟研究发现,基于可解释模型的决策能使学习者更快达到掌握状态;随后让12位真实教师基于不同模型提供的信息进行教学决策。结果显示,教师更认可可解释模型的可用性和可信度,但两种模型在达成掌握所需的练习任务数量上几乎没有差异。这表明模型可解释性与教师决策之间的关系并不直接:教师并非仅依赖模型做决策,未来需进一步研究学习者与教师如何真正理解和运用KT模型。
原文摘要 · Abstract (English)
Knowledge tracing (KT) models are a crucial basis for pedagogical decision-making, namely which task to select next for a learner and when to stop teaching a particular skill. Given the high stakes of pedagogical decisions, KT models are typically required to be interpretable, in the sense that they should implement an explicit model of human learning and provide explicit estimates of learners' abilities. However, to our knowledge, no study to date has investigated whether the interpretability of KT models actually helps human teachers to make teaching decisions. We address this gap. First, we perform a simulation study to show that, indeed, decisions based on interpretable KT models achieve mastery faster compared to decisions based on a non-interpretable model. Second, we repeat the study but ask $N=12$ human teachers to make the teaching decisions based on the information provided by KT models. As expected, teachers rate interpretable KT models higher in terms of usability and trustworthiness. However, the number of tasks needed until mastery hardly differs between KT models. This suggests that the relationship between model interpretability and teacher decisions is not straightforward: teachers do not solely rely on KT models to make decisions and further research is needed to investigate how learners and teachers actually understand and use KT models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。