arXiv:2602.17542cs.CLcs.CY2026-02

用大模型自动标注编程题中每个知识点的掌握情况,提升学习分析准确性。

Using LLMs for Knowledge Component-level Correctness Labeling in Open-ended Coding Problems

  • 利用大模型直接判断学生代码中每个知识点是否正确应用
  • 相比整体对错标注,学习曲线更符合认知理论,预测性能更好
  • 适合教育数据挖掘、自适应学习系统研究者使用

细粒度技能表示(即知识组件,KCs)是学生建模与学习分析的基础。然而,在开放性编程任务中,真实数据集很少提供KC级别的正确性标签,因为解法通常同时涉及多个知识点。简单将题目整体对错传播到所有相关知识点会掩盖部分掌握情况,导致学习曲线拟合不佳。为此,我们提出一种基于大语言模型(LLMs)的自动化框架,直接从学生代码中标注KC级别的正确性。方法通过评估每个知识点在代码中的正确应用,并引入时序感知的Code-KC映射机制,更准确地对齐知识点与个体代码。在幂律练习和加法因子模型下评估结果表明,该框架生成的学习曲线更符合认知理论,预测性能优于基线。人工评估显示,大模型标注与专家标注高度一致。

原文摘要 · Abstract (English)

Fine-grained skill representations, commonly referred to as knowledge components (KCs), are fundamental to many approaches in student modeling and learning analytics. However, KC-level correctness labels are rarely available in real-world datasets, especially for open-ended programming tasks where solutions typically involve multiple KCs simultaneously. Simply propagating problem-level correctness to all associated KCs obscures partial mastery and often leads to poorly fitted learning curves. To address this challenge, we propose an automated framework that leverages large language models (LLMs) to label KC-level correctness directly from student-written code. Our method assesses whether each KC is correctly applied and further introduces a temporal context-aware Code-KC mapping mechanism to better align KCs with individual student code. We evaluate the resulting KC-level correctness labels in terms of learning curve fit and predictive performance using the power law of practice and the Additive Factors Model. Experimental results show that our framework leads to learning curves that are more consistent with cognitive theory and improves predictive performance, compared to baselines. Human evaluation further demonstrates substantial agreement between LLM and expert annotations.

知识组件大模型编程教育学习分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。