用检索增强生成提升大模型科学问答评分准确率
Enhancing LLM-Based Short Answer Grading with Retrieval-Augmented Generation
- 根据题目和作答内容动态检索教育知识库
- 在科学教育数据集上评分准确率显著提升
- 适合教育AI评估系统研发者参考
短答案评估是科学教育中的关键环节，可检验学生对复杂三维知识的理解。具备类人语言能力的大模型正被用于辅助人工评分以减轻负担，但其领域知识有限，难以理解任务特定要求，导致性能受限。检索增强生成（RAG）通过在评分过程中接入相关领域知识，成为潜在解决方案。本文提出一种自适应RAG框架，基于问题与学生作答上下文动态检索并融合领域知识。该方法结合语义搜索与精选教育资料，实现高质量参考材料的获取。在科学教育数据集上的实验表明，本系统相比基线大模型方法，在评分准确率上取得显著提升。结果表明，RAG增强的评分系统能提供可靠且高效的性能改进。
原文摘要 · Abstract (English)
Short answer assessment is a vital component of science education, allowing evaluation of students' complex three-dimensional understanding. Large language models (LLMs) that possess human-like ability in linguistic tasks are increasingly popular in assisting human graders to reduce their workload. However, LLMs' limitations in domain knowledge restrict their understanding in task-specific requirements and hinder their ability to achieve satisfactory performance. Retrieval-augmented generation (RAG) emerges as a promising solution by enabling LLMs to access relevant domain-specific knowledge during assessment. In this work, we propose an adaptive RAG framework for automated grading that dynamically retrieves and incorporates domain-specific knowledge based on the question and student answer context. Our approach combines semantic search and curated educational sources to retrieve valuable reference materials. Experimental results in a science education dataset demonstrate that our system achieves an improvement in grading accuracy compared to baseline LLM approaches. The findings suggest that RAG-enhanced grading systems can serve as reliable support with efficient performance gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。