arXiv:2601.15626eess.SYcs.AI2026-01

用二元问题框架让AI精准评工程数学题,反馈更完整。

Bridging Qualitative Rubrics and AI: A Binary Question Framework for Criterion-Referenced Grading in Engineering

  • 用二元问题设计评估框架,让AI逐项判断学生答案对错
  • AI评分准确率达92.5%,接近两名专家水平
  • 适合希望提升批改效率与反馈质量的工程教育者

本研究探讨生成式AI如何与基于标准的评分体系结合,提升工程类数学测验的评分效率与质量。针对助教在基于模型解法的手动评分中面临的挑战,研究设计了一种由生成式AI支持的系统,可可靠识别学生错误、提供高质量反馈,并辅助人工评分。实验结果显示,该系统整体评分准确率达92.5%,与两名经验丰富的助教水平相当。两位研究人员(亦为助教)认为,该系统作为第二评审者显著提升了评分准确性,能发现细微错误并提供更全面的反馈。核心成果是实现高质量、可扩展的形成性反馈嵌入评估流程。但研究也指出,该工具尚不足以独立使用,尤其在应对非标准解法时可靠性不足。未来需进一步研究学生对AI评分与反馈的感知。

原文摘要 · Abstract (English)

PURPOSE OR GOAL: This study investigates how GenAI can be integrated with a criterion-referenced grading framework to improve the efficiency and quality of grading for mathematical assessments in engineering. It specifically explores the challenges demonstrators face with manual, model solution-based grading and how a GenAI-supported system can be designed to reliably identify student errors, provide high-quality feedback, and support human graders. The research also examines human graders' perceptions of the effectiveness of this GenAI-assisted approach. ACTUAL OR ANTICIPATED OUTCOMES: The study found that GenAI achieved an overall grading accuracy of 92.5%, comparable to two experienced human graders. The two researchers, who also served as subject demonstrators, perceived the GenAI as a helpful second reviewer that improved accuracy by catching small errors and provided more complete feedback than they could manually. A central outcome was the significant enhancement of formative feedback. However, they noted the GenAI tool is not yet reliable enough for autonomous use, especially with unconventional solutions. CONCLUSIONS/RECOMMENDATIONS/SUMMARY: This study demonstrates that GenAI, when paired with a structured, criterion-referenced framework using binary questions, can grade engineering mathematical assessments with an accuracy comparable to human experts. Its primary contribution is a novel methodological approach that embeds the generation of high-quality, scalable formative feedback directly into the assessment workflow. Future work should investigate student perceptions of GenAI grading and feedback.

AI评分工程教育生成式AI反馈机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。