用提示工程提升大模型对电路分析作业的评分准确率
Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis
- 通过多步提示与上下文数据增强改进模型推理
- 评分准确率从74.71%提升至97.70%
- 适合工程教育领域自动化评估场景
本研究提出一套增强大型语言模型（LLM）在本科生电路分析课程作业评估中的方法，旨在提升模型为电气工程学生提供个性化支持的能力。已有评估表明，GPT-4o 在该领域具备出色的作业评分潜力。基于此，我们通过多步提示、上下文数据增强和引入针对性提示，进一步优化 GPT-4o 表现。这些策略有效解决了简单提示下模型常见的错误，显著提升了评估准确性。具体而言，在基础电路分析主题上，经增强提示与数据扩充后，GPT-4o 的正确响应率从 74.71% 提升至 97.70%。该工作为大模型在电路分析教学中的有效整合奠定了基础，也推动了其在工程教育中的广泛应用。
原文摘要 · Abstract (English)
This research full paper presents an enhancement pipeline for large language models (LLMs) in assessing homework for an undergraduate circuit analysis course, aiming to improve LLMs' capacity to provide personalized support to electrical engineering students. Existing evaluations have demonstrated that GPT-4o possesses promising capabilities in assessing student homework in this domain. Building on these findings, we enhance GPT-4o's performance through multi-step prompting, contextual data augmentation, and the incorporation of targeted hints. These strategies effectively address common errors observed in GPT-4o's responses when using simple prompts, leading to a substantial improvement in assessment accuracy. Specifically, the correct response rate for GPT-4o increases from 74.71% to 97.70% after applying the enhanced prompting and augmented data on entry-level circuit analysis topics. This work lays a foundation for the effective integration of LLMs into circuit analysis instruction and, more broadly, into engineering education.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。