用测试用例增强代码理解,精准追踪编程学习者知识掌握情况
Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks
- 结合测试用例与学生代码,多任务学习建模
- 在CodeWorkout数据集上显著优于仅依赖总分的模型
- 适合教育数据分析与个性化学习系统开发者
开放性编程任务是计算机科学教育中的常见题型,要求学生根据需求编写程序。由于学生代码形式多样,传统知识追踪(KT)模型仅依赖答题正确性难以捕捉真实知识状态。本文提出TIKTOC框架,通过引入部分题目对应的测试用例,同时分析并预测:1)学生代码是否通过每个测试用例;2)学生提交的完整代码。我们在CodeWorkout数据集上扩充了测试用例,并以大语言模型为骨干,设计多任务学习方法。定量结果显示,该方法显著优于仅使用代码总分的现有KT模型。定性分析表明,结合测试用例信息与代码内容,可获得更精细的学生知识理解。
原文摘要 · Abstract (English)
Open-ended coding tasks, which ask students to construct programs according to certain specifications, are common in computer science education. Student modeling can be challenging since their open-ended nature means that student code can be diverse. Traditional knowledge tracing (KT) models that only analyze response correctness may not fully capture nuances in student knowledge from student code. In this paper, we introduce Test case-Informed Knowledge Tracing for Open-ended Coding (TIKTOC), a framework to simultaneously analyze and predict both open-ended student code and whether the code passes each test case. We augment the existing CodeWorkout dataset with the test cases used for a subset of the open-ended coding questions, and propose a multi-task learning KT method to simultaneously analyze and predict 1) whether a student's code submission passes each test case and 2) the student's open-ended code, using a large language model as the backbone. We quantitatively show that these methods outperform existing KT methods for coding that only use the overall score a code submission receives. We also qualitatively demonstrate how test case information, combined with open-ended code, helps us gain fine-grained insights into student knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。