用大模型模拟不同认知水平的学生,让错误更真实。
Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents
- 基于知识图谱构建学生认知原型,预测学习表现
- 通过束搜索迭代修正答案,还原真实学习错误
- 在100名学生5000条记录上实现模拟准确率提升100%
大型语言模型正重塑教育领域,其代理在模拟学生行为中扮演关键角色。当前主要挑战在于如何建模不同认知水平学生的多样化学习模式。然而,主流LLM通常训练为‘助人助手’,倾向于生成完美答案,难以模拟具有真实认知缺陷的学习者,导致仿真结果失真。为此,本文提出无需训练的仿真框架:首先利用知识图谱构建每位学生的认知原型,捕捉其过往学习记录中的概念掌握情况;再将该原型映射至新任务以预测表现;随后基于预测结果生成学生解答,并采用束搜索方法迭代优化,更真实还原学习过程中的典型错误。为验证方法有效性,我们构建了包含100名学生和5000条学习记录的 exttt{Student\_100}数据集。实验表明,本方法在仿真准确率上相较基线模型实现100%提升。
原文摘要 · Abstract (English)
Large language models (LLMs) are revolutionizing education, with LLM-based agents playing a key role in simulating student behavior. A major challenge in student simulation is modeling the diverse learning patterns of students at various cognitive levels. However, current LLMs, typically trained as ``helpful assistants'', target at generating perfect responses. As a result, they struggle to simulate students with diverse cognitive abilities, as they often produce overly advanced answers, missing the natural imperfections that characterize student learning and resulting in unrealistic simulations. To address this issue, we propose a training-free framework for student simulation. We begin by constructing a cognitive prototype for each student using a knowledge graph, which captures their understanding of concepts from past learning records. This prototype is then mapped to new tasks to predict student performance. Next, we simulate student solutions based on these predictions and iteratively refine them using a beam search method to better replicate realistic mistakes. To validate our approach, we construct the \texttt{Student\_100} dataset, consisting of $100$ students working on Python programming and $5,000$ learning records. Experimental results show that our method consistently outperforms baseline models, achieving $100\%$ improvement in simulation accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。