用大模型模拟学生错解,还能正确解题。
LLM-based Cognitive Models of Students with Misconceptions
- 通过指令微调大模型,模仿学生真实错解模式。
- 训练中错例占比低至0.25时,模型既会错又会正解。
- 适合开发自适应教育系统,研究认知偏差的学者可参考。
准确建模学生认知对发展AI驱动的教育技术至关重要。核心挑战在于构建满足双重特性的学生模型:(1)精准复现特定错误概念,(2)在错误不适用的问题上仍能正确求解。这反映了学生理解中错误与正确认知并存的复杂性。本文探究大语言模型(LLMs)是否可通过指令微调实现这一双重目标,并应用于代数学习。我们提出MalAlgoPy——一个基于图结构表示代数问题求解过程的Python库,用于生成反映真实学生解题模式的数据集。基于此,定义并研究认知学生模型(CSMs)——经指令微调后忠实模拟学生行为的LLM。研究发现,仅用错误示例训练的模型能高效复现错误;但训练会削弱其在无错误场景下的解题能力,无法满足第二特性。我们证明,通过精细调控训练数据中正确与错误示例的比例(低至0.25),可构建同时满足两性质的CSMs。研究深化了对基于AI的学生模型的理解,为有效自适应学习系统奠定基础。
原文摘要 · Abstract (English)
Accurately modeling student cognition is crucial for developing effective AI-driven educational technologies. A key challenge is creating realistic student models that satisfy two essential properties: (1) accurately replicating specific misconceptions, and (2) correctly solving problems where these misconceptions are not applicable. This dual requirement reflects the complex nature of student understanding, where misconceptions coexist with correct knowledge. This paper investigates whether Large Language Models (LLMs) can be instruction-tuned to meet this dual requirement and effectively simulate student thinking in algebra. We introduce MalAlgoPy, a novel Python library that generates datasets reflecting authentic student solution patterns through a graph-based representation of algebraic problem-solving. Utilizing MalAlgoPy, we define and examine Cognitive Student Models (CSMs) - LLMs instruction tuned to faithfully emulate realistic student behavior. Our findings reveal that LLMs trained on misconception examples can efficiently learn to replicate errors. However, the training diminishes the model's ability to solve problems correctly, particularly for problem types where the misconceptions are not applicable, thus failing to satisfy second property of CSMs. We demonstrate that by carefully calibrating the ratio of correct to misconception examples in the training data - sometimes as low as 0.25 - it is possible to develop CSMs that satisfy both properties. Our insights enhance our understanding of AI-based student models and pave the way for effective adaptive learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。