不依赖语言模型，用简单信号提升眼动预测阅读理解能力
LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity
- 在眼动数据中注入词级难度信号，通过拼接或残差机制增强模型
- 在未见文本上提升准确率1.8至2.9个百分点，统计显著
- 适合关注眼动分析与轻量化模型的读者，尤其关注跨读者泛化
在最近的EyeBench基准上，基于预训练语言模型的文本感知模型阅读理解预测达到56%-63% AUROC，而仅使用眼动数据的模型表现仅为随机水平。本文探索无需语言模型的轻量级条件化方法能否提升眼动模型性能。基于EyeBench的AhnCNN基线，提出LEXIC-Base，通过两种机制将三个预计算的词级难度信号（GPT-2困惑度、词频、词长）注入每注视点输入：直接拼接（LEXIC-Concat）和残差机制（LEXIC-Res），后者由小头预测典型读者眼动响应，编码器基于偏差进行条件化。在OneStop阅读理解任务中，采用K=5种子集成训练十折交叉验证，两种机制在未见文本上均实现+1.8至+2.2百分点的AUROC提升（Wilcoxon p ≤ 0.065）。LEXIC-Concat进一步在未见读者上提升+2.9百分点（p = 0.010）。LEXIC-Res在未见读者上提升+1.8百分点（p = 0.19），其上限源于预测头在训练读者上校准，跨分布泛化不佳。
原文摘要 · Abstract (English)
On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance. We ask how far a gaze-only model can be pushed by lightweight, language-model-free conditioning. Building on the EyeBench AhnCNN baseline, LEXIC-Base, we propose two mechanisms to inject three precomputed word-level difficulty signals, GPT-2 surprisal, word frequency, and word length, into the per-fixation input: direct concatenation, LEXIC-Concat, and a residual mechanism, LEXIC-Res, where a small head predicts typical-reader gaze response and the encoder is conditioned on the deviation. On the OneStop reading comprehension task, with K=5 seed-ensemble training across ten folds, both mechanisms produce statistically consistent AUROC gains on Unseen Text, +1.8 to +2.2 percentage points, Wilcoxon p <= 0.065. LEXIC-Concat additionally lifts Unseen Reader by +2.9 percentage points, p = 0.010. We trace an architectural boundary in LEXIC-Res on Unseen Reader, +1.8 percentage points, p = 0.19, to the prediction head being calibrated to training readers, transferring imperfectly to out-of-distribution readers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。