用大模型评估编程自解释，比传统语义相似度更准。
Exploring the Effectiveness of Using LLMs for Automated Assessment of Student Self Explanations in Programming Education
- 用大模型直接判断学生编程解释对错
- 大模型在准确率上显著优于语义匹配方法
- 适合教育技术研究者和智能评测系统开发者
示范例题是特定领域中逐步解决问题的范例，用于帮助学生掌握问题解决技能。若将示范例题与自解释结合，即要求学生主动解释每一步解题逻辑，可进一步提升学习效果。但该方法的主要挑战在于如何自动评估学生解释的正确性。目前主流方法依赖学生答案与教师或专家解释之间的语义相似度进行评分。随着大模型在自动评分中的进展，尚不清楚语义相似度是否仍是最佳方案。此外，此类比较需要具备平衡类别分布和领域标注数据的高质量数据集。本文针对自动化评分任务，开展大模型与语义相似度方法的严格对比，将其建模为二分类问题。
原文摘要 · Abstract (English)
Worked examples are step-by-step solutions to problems in a specific domain, offered to students to acquire domain-specific problem-solving skills. The effectiveness of worked examples could be enhanced by combining them with self-explanations, which ask students to explain rather than passively study each problem-solving step. The main challenge of this approach is assessing the correctness of the student's explanations. In the prevailing approach, student explanations are judged by their semantic similarity to an instructor's or domain expert's explanation. Given recent advances in LLM-based automated scoring, it remains unclear whether semantic similarity methods are still the most effective technique to automatically score textual student responses like essays or code explanations. Comparing these methods also requires quality datasets that offer distinctive features such as balanced class distributions and domain-specific labeled data for automated scoring tasks. In this paper, we present a rigorous comparison between LLMs and semantic similarity used for automated scoring, framed as a binary classification task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。