用图结构分析阅读理解，发现大模型与人类认知高度一致。
Graph Representations for Reading Comprehension Analysis using Large Language Model and Eye-Tracking Biomarker
- 将文本转为语义图，节点为词组，边为语义关联。
- 眼动数据表明，重要节点和边的注视分布与模型一致。
- 适合研究人机认知对比或AI辅助教育场景。
阅读理解是人类认知发展的基础能力。随着大语言模型（LLMs）的发展，亟需比较人类与LLMs在不同情境下的语言理解差异，并将其应用于推理、情感解读和信息检索等任务。此前工作利用LLMs和人类生物标志物研究阅读理解过程，结果表明：由LLMs标记为与推理目标高/低相关性的词，其对应的眼动特征呈现明显差异。但仅关注单个词限制了理解深度，结论相对简单。本研究通过基于LLM的智能体，将阅读文本中的词语分组形成节点与边，构建基于语义和问题导向提示的图结构文本表示。随后对比重要节点和边上的人眼固定点分布。结果表明，LLMs在图拓扑结构层面的语言理解具有高度一致性。该发现拓展了先前研究，为有效的人机协同学习策略提供启示。
原文摘要 · Abstract (English)
Reading comprehension is a fundamental skill in human cognitive development. With the advancement of Large Language Models (LLMs), there is a growing need to compare how humans and LLMs understand language across different contexts and apply this understanding to functional tasks such as inference, emotion interpretation, and information retrieval. Our previous work used LLMs and human biomarkers to study the reading comprehension process. The results showed that the biomarkers corresponding to words with high and low relevance to the inference target, as labeled by the LLMs, exhibited distinct patterns, particularly when validated using eye-tracking data. However, focusing solely on individual words limited the depth of understanding, which made the conclusions somewhat simplistic despite their potential significance. This study used an LLM-based AI agent to group words from a reading passage into nodes and edges, forming a graph-based text representation based on semantic meaning and question-oriented prompts. We then compare the distribution of eye fixations on important nodes and edges. Our findings indicate that LLMs exhibit high consistency in language understanding at the level of graph topological structure. These results build on our previous findings and offer insights into effective human-AI co-learning strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。