无需标签的多主题作文自动评分系统,提升复杂答题的准确性
GRASP: Graph-Retrieval Automated Scoring Pipeline for Label-Free Multi-Topic Essay Grading
- 用图检索技术从参考答案中定位匹配内容,不依赖训练数据
- 在复杂多主题作文中,评分准确率比纯相似度方法提升12.7%
- 适合教育评测、智能阅卷场景,尤其适用于无标签混合答题
自动短答案评分研究长期聚焦于单一主题的考试。针对包含多个主题的混合式短答考试,本文提出一种无需标签的多主题科学考试自动评分流水线GRASP。学生将不同主题的回答合并成一段无标记文本。通过Sentence-BERT将各题参考答案编码进FAISS向量索引,并构建语义相似性图。评分时,先利用句数启发式与大语言模型推断学生作答的主题数量,无需训练数据。随后通过基于余弦相似度的RAG和图增强检索GRAG获取候选参考节点。GRAG以高相似度节点为种子,沿强边进行图遍历,补充遗漏节点。最后使用匈牙利算法将每个题目段落最优匹配一个参考节点,再由GPT-4.1-mini独立评分。实验验证了检索质量对评分精度的影响,并证明在不同复杂度下,图增强检索优于传统余弦相似方法。
原文摘要 · Abstract (English)
Automated short-answer grading research has historically focused on exams consisting solely of questions pertaining to a single topic. Automatic grading of exams containing questions about more than one topic remains less explored. In this work, a Graph-Retrieval Automated Scoring Pipeline (GRASP) is introduced for grading label-free multi-topic science exams. Label-free exams are short-answer exams in which a student's responses to several distinct topics are merged into a single paragraph, with no markup labels or segmentation indicating which span answers which question. Reference answers for each question are encoded into a FAISS vector index via Sentence-BERT, and a semantic similarity graph is constructed over this set of reference answers. At grading time, sentence count heuristics, with a large language model used to resolve ambiguous cases, are first applied to predict how many distinct topics were answered in the student essay. This process is performed without training data or domain-specific example essays. Candidate reference nodes, each storing one (question, reference answer, concatenation of both) from the reference index, are then retrieved through cosine similarity based Retrieval-Augmented Generation (RAG) and Graph Retrieval-Augmented Generation (GRAG). GRAG operates by taking the top cosine matches as seed nodes and then performing a graph traversal over strong edges to find additional reference nodes that may have been missed by RAG. The Hungarian algorithm is then used to optimally assign one reference node per question segment such that no reference is duplicated. Each segment is then graded against its assigned reference independently using GPT-4.1-mini. This experiment is performed to show the effect of retrieval quality on grading accuracy and the benefit of graph-augmented retrieval versus strict cosine similarity methods at various levels of essay complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。