用轻量模型生成逼真阅读眼动轨迹,助力语言理解研究
Eyettention II: A Dual-Sequence Architecture for Modeling Fixation Location, Within-Word Landing Position, and Fixation Duration in Reading

- 双序列架构同步预测注视位置、词内落点和持续时间
- 在有限数据下仍超越现有模型,符合人类认知规律
- 适合语言模型优化与心理语言学实验设计
阅读时眼动行为揭示了读者认知过程与文本特征的深层关系。眼动追踪数据在增强语言模型、推断读者特征等技术应用中具有重要价值,但其依赖大规模数据,而数据采集成本高、难度大。为此,我们提出Eyettention II——一个端到端训练的轻量级深度学习模型,可生成包含注视位置、词内落点及注视持续时间的完整眼动轨迹。该模型高效可训,仅需少量GPU资源,且与认知理论高度契合。实验表明,Eyettention II在眼动轨迹预测上优于当前最优模型,能捕捉关键心理语言学现象,模拟人类注视行为。其优异性能有望推动自然语言处理发展,支持心理语言学实验材料预研,并揭示传统认知模型未涵盖的新洞见。
原文摘要 · Abstract (English)
The way our eyes move while reading provides valuable insights into both the reader's cognitive processes and the properties of the text. In particular, eye-tracking-while-reading data has shown to be highly beneficial in various technological applications, such as enhancing and interpreting language models and inferring a reader's characteristics. However, these applications often rely on large-scale, data-driven models, which demand extensive eye-tracking datasets that are challenging to obtain due to the resource-intensive nature of data collection. To address the challenge of data scarcity, we develop Eyettention II, an end-to-end trained deep-learning model capable of generating realistic scanpaths consisting of a complete set of fixation attributes in chronological order, including fixation location, within-word landing position, and fixation duration. Our model is lightweight, efficiently trainable on limited GPU resources, and closely aligned with cognitive theories. We demonstrate that Eyettention II surpasses state-of-the-art models in scanpath prediction and mirrors human-like gaze behavior by capturing key psycholinguistic phenomena. With its robust performance, Eyettention II holds the potential to drive advancements in natural language processing, facilitate piloting the materials of psycholinguistic experiments, and uncover new insights beyond what is explicitly encoded in theoretical cognitive models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。