用BART模型结合评分标准,让自动评分更贴近老师打分习惯。
Leveraging BART to Assess CS1 C++ Programming Assignments using Rubric-based Criteria

- 用多任务学习和软标签设计,让模型同时预测分数和等级。
- 相比传统方法,评分分布更接近真实教师打分,误差更低。
- 适合需要精准、可解释自动评分的编程教学场景。
本文研究基于评分标准的多任务微调变压器模型,用于自动评估入门级C++编程作业,目标是使评分结果更贴近教师实际打分行为。利用多学期的CS1课程数据,学生提交代码与数值分数、等级区间及评分细则配对,并预处理为统一序列输入模型。采用带有LoRA适配的BART编码器-解码器架构,联合预测数值分数与等级区间,引入分布匹配项以对齐预测与实际评分分布,这一维度在以往工作中常被忽略。实验对比了单任务与多任务训练、硬标签与模糊/边界软标签、有无评分标准等设置,还包括T5和成对预训练变体。结果显示,采用边界软标签和评分上下文的多任务BART模型,在均方绝对误差和评分分布一致性上均优于单任务、硬标签或仅代码基线。全微调T5进一步提升了分布保真度,而成对预训练虽降低数值误差,但牺牲了少数类敏感性。总体表明,具备校准意识且受评分标准引导的训练方式,能生成更像人类教师的评分行为。
原文摘要 · Abstract (English)
This paper investigates rubric-aware, multitask fine-tuning of transformer models for automated grading of introductory C++ programming assignments, with the goal of producing grade predictions that better reflect instructor grading behavior than general-purpose LLMs. Using multi-semester CS1 data, student submissions are paired with numeric scores, letter-grade buckets, and assignment rubrics, then preprocessed into unified sequences for transformer input. A BART encoder-decoder with LoRA adaptation is trained to jointly predict numeric grades and grade buckets, augmented with a distribution-matching term to align predicted and empirical grade distributions, an evaluation dimension often overlooked in prior work. Experiments compare single-task and multitask training, hard one-hot versus fuzzy and boundary-based soft labels, and rubric versus no-rubric conditions, with additional T5 and pairwise-pretrained variants. Results show that multitask BART with boundary-based soft labels and rubric context achieves lower mean absolute error and stronger grade-distribution alignment than single-task, hard-label, or code-only baselines. Fully fine-tuned T5 further improves distributional fidelity, while pairwise pretraining reduces numeric error at the cost of minority-class sensitivity. Collectively, the findings suggest that calibration-aware, rubric-guided training produces more instructor-like grading behavior than accuracy-optimized alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。