用知识图谱提升自动作文评分,让模型更懂逻辑推理。
From Flat to Structural: Enhancing Automated Short Answer Grading with GraphRAG
- 将参考材料构建成知识图谱,显式建模概念间关系。
- 在NGSS数据集上,对科学实践的评分准确率显著提升。
- 适合需要逻辑链验证的高阶教育评估场景。
自动化短答案评分(ASAG)对规模化教育评估至关重要,但大语言模型(LLMs)因依赖通用预训练,常出现幻觉且难以严格遵循评分标准。尽管检索增强生成(RAG)可缓解此问题,但传统的“扁平”向量检索将知识视为孤立片段,无法捕捉复杂教育内容中的结构关系与多跳推理需求。为此,我们提出图检索增强生成(GraphRAG)框架,将参考材料组织为结构化知识图谱,显式建模概念间的依赖关系。方法采用双阶段流程:利用Microsoft GraphRAG构建高保真图谱,再通过HippoRAG神经符号算法执行关联图遍历,从而检索出完整、连贯的证据子图。在下一代科学标准(NGSS)数据集上的实验表明,该结构化方法在所有指标上均显著优于标准RAG基线。尤其在科学与工程实践(SEP)评估中,HippoRAG实现显著提升,证实了结构化检索在验证高阶学术评估所需逻辑推理链方面的优势。
原文摘要 · Abstract (English)
Automated short answer grading (ASAG) is critical for scaling educational assessment, yet large language models (LLMs) often struggle with hallucinations and strict rubric adherence due to their reliance on generalized pre-training. While Rretrieval-Augmented Generation (RAG) mitigates these issues, standard "flat" vector retrieval mechanisms treat knowledge as isolated fragments, failing to capture the structural relationships and multi-hop reasoning essential for complex educational content. To address this limitation, we introduce a Graph Retrieval-Augmented Generation (GraphRAG) framework that organizes reference materials into a structured knowledge graph to explicitly model dependencies between concepts. Our methodology employs a dual-phase pipeline: utilizing Microsoft GraphRAG for high-fidelity graph construction and the HippoRAG neurosymbolic algorithm to execute associative graph traversals, thereby retrieving comprehensive, connected subgraphs of evidence. Experimental evaluations on a Next Generation Science Standards (NGSS) dataset demonstrate that this structural approach significantly outperforms standard RAG baselines across all metrics. Notably, the HippoRAG implementation achieved substantial improvements in evaluating Science and Engineering Practices (SEP), confirming the superiority of structural retrieval in verifying the logical reasoning chains required for higher-order academic assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。