用AI自动批改理工科多模态答题卷,文本与图表一并评估。
Automated Assessment of Multimodal Answer Sheets in the STEM domain
- 用大模型将手绘流程图转为文本进行语义评估
- 结合文本抽取与目标检测模型,实现图文联合评分
- 适合需要高效批改理工科作业的教育机构
在教育领域,技术融合正重塑传统教学模式。尤其在科学、技术、工程和数学(STEM)领域,自动化评分面临独特挑战,涵盖从定量分析到手写图表解读等多个方面。本文提出一种基于人工智能的自动化评估方法,旨在提升评分效率与可靠性。首先,利用样本答案与自然语言处理技术,构建精准文本答案评估系统;其次,针对流程图等图表内容,通过将视觉信息转化为文本表示,并借助大语言模型(LLM)进行细致语义分析,实现对图形表达的准确评估。研究整合CRAFT文本提取模型、YoloV5目标检测模型以及Mistral-7B大语言模型,实现对多模态答题卷的全面评估。实验表明,该方法显著降低人工干预,提升评分一致性与可扩展性,为推动人工智能在理工科教育评价中的应用提供可行路径。
原文摘要 · Abstract (English)
In the domain of education, the integration of,technology has led to a transformative era, reshaping traditional,learning paradigms. Central to this evolution is the automation,of grading processes, particularly within the STEM domain encompassing Science, Technology, Engineering, and Mathematics.,While efforts to automate grading have been made in subjects,like Literature, the multifaceted nature of STEM assessments,presents unique challenges, ranging from quantitative analysis,to the interpretation of handwritten diagrams. To address these,challenges, this research endeavors to develop efficient and reliable grading methods through the implementation of automated,assessment techniques using Artificial Intelligence (AI). Our,contributions lie in two key areas: firstly, the development of a,robust system for evaluating textual answers in STEM, leveraging,sample answers for precise comparison and grading, enabled by,advanced algorithms and natural language processing techniques.,Secondly, a focus on enhancing diagram evaluation, particularly,flowcharts, within the STEM context, by transforming diagrams,into textual representations for nuanced assessment using a,Large Language Model (LLM). By bridging the gap between,visual representation and semantic meaning, our approach ensures accurate evaluation while minimizing manual intervention.,Through the integration of models such as CRAFT for text,extraction and YoloV5 for object detection, coupled with LLMs,like Mistral-7B for textual evaluation, our methodology facilitates,comprehensive assessment of multimodal answer sheets. This,paper provides a detailed account of our methodology, challenges,encountered, results, and implications, emphasizing the potential,of AI-driven approaches in revolutionizing grading practices in,STEM education.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。