用380万次编程编辑痕迹训练模型，捕捉学生编程思维与纠错路径。
Modeling Student Learning with 3.8 Million Program Traces
- 基于真实编程编辑轨迹训练语言模型，还原学生思考过程。
- 模型可预测学生行为特征，如回溯次数、注释数量等，准确率超基准。
- 能生成贴近学生风格的修正代码，帮助其自主纠错。
程序员在编写代码时频繁修改和重试，形成丰富的“交互轨迹”，揭示其解题思路与技能发展水平。对于初学者而言，这些轨迹反映其探索性学习、调试策略调整及个人编码风格等多样化认知过程。本文构建了一个来自Pencil Code平台（一个免费在线编程教育平台）的超过380万条编程推理轨迹数据集。相比仅使用最终代码或合成轨迹训练的模型，基于真实轨迹训练的模型在建模学生多样行为方面表现更优。通过行为分析与探针测试发现，代码轨迹中的诸多属性（如目标回溯次数、注释数量）可从学生表征中有效预测。基于此，我们进一步展示了如何引导代码生成模型识别一系列编辑操作，使代码更正确的同时保持与学生原始风格一致。结果表明，许多代码特征本质上是学生的个体属性，而基于编辑轨迹训练的模型具备更强的可调控性、对学生行为的预测能力，以及生成最终代码的能力。代码与数据已公开于https://github.com/meghabyte/pencilcode-public。
原文摘要 · Abstract (English)
As programmers write code, they often edit and retry multiple times, creating rich "interaction traces" that reveal how they approach coding tasks and provide clues about their level of skill development. For novice programmers in particular, these traces reflect the diverse reasoning processes they employ to code, such as exploratory behavior to understand how a programming concept works, re-strategizing in response to bugs, and personalizing stylistic choices. In this work, we explore what can be learned from training language models on such reasoning traces: not just about code, but about coders, and particularly students learning to program. We introduce a dataset of over 3.8 million programming reasoning traces from users of Pencil Code, a free online educational platform used by students to learn simple programming concepts. Compared to models trained only on final programs or synthetically-generated traces, we find that models trained on real traces are stronger at modeling diverse student behavior. Through both behavioral and probing analyses, we also find that many properties of code traces, such as goal backtracking or number of comments, can be predicted from learned representations of the students who write them. Building on this result, we show that we can help students recover from mistakes by steering code generation models to identify a sequence of edits that will results in more correct code while remaining close to the original student's style. Together, our results suggest that many properties of code are properties of individual students and that training on edit traces can lead to models that are more steerable, more predictive of student behavior while programming, and better at generating programs in their final states. Code and data is available at https://github.com/meghabyte/pencilcode-public
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。