发现语言模型不同层对人类句子处理有双重匹配现象
Dual Alignment Between Language Model Layers and Human Sentence Processing
- 对比自然阅读与句法挑战场景，不同层模型更匹配人类认知
- 后期层在复杂句式中更接近人类认知努力，但仍低估真实数据
- 结合浅层与深层概率更新可更好预测阅读时间
最近研究（Kuribayashi等，2025）表明，使用大型语言模型（LLM）早期层的意外度（surprisal）可有效建模人类在语法简单结构下的句子处理行为。这引发疑问：这种内部层的优势是否能延伸到更具句法挑战性的结构？在这些结构中，意外度曾被报告为低估了人类的认知努力。本文首先探究在英语句法歧义处理中，哪些模型层能更好地估计人类认知努力。实验显示，在自然阅读中，早期层更匹配；而在句法挑战情境下，后期层表现更好，但依然低估人类数据。这一双重对齐揭示了人类与模型在句子处理上的不同模式：自然阅读类似于模型早期层的弱预测，而复杂句式处理则依赖更充分上下文表示，由模型后期层更好建模。基于此，我们还探索了利用浅层与深层概率更新的多种测量方式，结果显示其组合优于单一层次的意外度，在阅读时间建模中具有互补优势。
原文摘要 · Abstract (English)
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。