arXiv:2608.21668cs.AIcs.HC2026-08

用随机学生知识图谱让大模型更真实地模拟不同水平学生答题。

From Mastery Profile to Simulated Response: Stochastic Student Knowledge Graphs (SSKG) for Faithful LLM Student Simulation

  • 基于知识图谱为每个知识点分配掌握概率,决定答题正确性。
  • 模拟准确率降至44.1%-85.2%,明显区分高低水平学生。
  • 适合用于生成可信学生数据或测试智能辅导系统。

大语言模型(LLMs)被广泛用于模拟不同掌握水平的学生,以生成合成训练数据或压力测试辅导系统。然而,常见的提示驱动方法将答案决策权交给模型自身,导致即使被指令模拟低掌握水平学生,模型仍按其固有能力表现,难以区分不同掌握水平。我们在379道经过大学理事会校准的SAT代数题和五个典型掌握水平上验证了这一问题:三个厂商的LLM(Gemini 3.1 Flash Lite、Claude Haiku 4.5、GPT-5.4-mini)在所有水平上的准确率均达96.8%-100%。为此,我们提出基于随机学生知识图谱(SSKG)的方法。从开源代数教材中提取课程知识图谱(CKG),并将每道SAT题的解法分解为一系列所需的知识三元组。SSKG为每个三元组分配掌握概率,通过采样决定题目是否答对,再由LLM生成与结果一致的第一人称推理过程。该方法使模拟准确率下降至44.1%-85.2%,并展现出清晰的掌握水平梯度。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used to simulate students at different mastery levels. These simulations can generate synthetic training data and stress-test tutoring systems. However, common prompt-based approaches leave the answer decision to the LLM, which tends to perform according to its built-in capabilities even when instructed to simulate a student with low mastery. As a result, these approaches may have difficulty distinguishing students with low and high levels of mastery. We demonstrate this limitation using 379 College Board-calibrated SAT Algebra items and five archetypal mastery profiles. Three LLMs from three vendors (Gemini 3.1 Flash Lite, Claude Haiku 4.5, and GPT-5.4-mini) achieve 96.8-100% accuracy across all profiles. To address this limitation, we introduce a method grounded in a Stochastic Student Knowledge Graph (SSKG). A curriculum knowledge graph (CKG) is extracted from an open algebra textbook, and each SAT solution is decomposed into a chain of required triples. The SSKG assigns a mastery probability to each triple, which is sampled to determine question correctness. An LLM then generates a first-person rationale consistent with the outcome. The simulation reduces accuracy to 44.1-85.2% across profiles and produces a clear monotone mastery gradient.

学生模拟知识图谱大模型教育技术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。