从阅读眼动数据中自动解析读者的开放性求知目标。
Decoding Open-Ended Information Seeking Goals from Eye Movements in Reading
- 基于眼动与文本的多模态模型，首次实现阅读目标解码。
- 在多个选项中准确识别正确目标，初步实现目标文本重建。
- 适用于教育辅助与实时阅读理解技术开发。
阅读时，人们常对文本中的特定信息抱有好奇心，例如对大语言模型在眼动研究中的应用、实验设计细节，或质疑其是否真实有效。日常阅读中，读者往往带有多种文本相关的求知目标，这些目标驱动其阅读行为。本文首次探讨：能否仅通过阅读时的眼动数据，自动解码开放式的阅读目标？为此，我们构建了基于大规模英文阅读眼动数据的目标解码任务与评估框架，涵盖数百个具体的求知任务。我们开发并比较了多种判别式与生成式多模态模型（融合文本与眼动数据），利用大语言模型进行目标识别与重构。实验表明，模型在多项选择任务中表现显著，且在自由形式的目标文本重建上已取得进展。该成果为探究目标驱动阅读的科学机制提供了新路径，并推动教育与辅助技术向实时解码读者意图发展。
原文摘要 · Abstract (English)
When reading, we often have specific information that interests us in a text. For example, you might be reading this paper because you are curious about LLMs for eye movements in reading, the experimental design, or perhaps you wonder ``This sounds like science fiction. Does it actually work?''. More broadly, in daily life, people approach texts with any number of text-specific goals that guide their reading behavior. In this work, we ask, for the first time, whether open-ended reading goals can be automatically decoded solely from eye movements in reading. To address this question, we introduce goal decoding tasks and evaluation frameworks using large-scale eye tracking for reading data in English with hundreds of text-specific information seeking tasks. We develop and compare several discriminative and generative multimodal text and eye movements LLMs for these tasks. Our experiments show considerable success on the task of selecting the correct goal among several options, and even progress towards free-form textual reconstruction of the precise goal formulation. These results open the door for further scientific investigation of goal driven reading, as well as the development of educational and assistive technologies that will rely on real-time decoding of reader goals from their eye movements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。