揭秘自回归Transformer如何逐步理解句子，发现其依赖语法与启发式规则并存。
Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models
- 用稀疏自编码器识别语言模型处理句子时的关键特征。
- 多数特征关联语法结构，部分反映无关语法的启发式判断。
- 模型同时考虑多种解释，但不修正初始错误理解。
自回归Transformer语言模型具备强大的句法能力，通常能成功处理从一致到否定词限定等现象。然而，它们用于逐步处理语言输入的特征尚不明确。本文通过研究花园路径句的处理机制来填补这一空白。我们提出三个问题：（1）语言模型是使用句法特征还是浅层启发式规则进行增量处理？（2）模型是否仅表征一种可能的解释，还是多种？（3）模型是否会重新分析或修复初始错误表征？为此，我们使用稀疏自编码器识别决定花园路径句后续选择（即阅读方式）的可解释特征。结果发现，虽然许多重要特征与句法结构相关，但也存在反映语义无关启发式规则的特征。此外，尽管大多数活跃特征对应一个句子解释，部分特征却对应另一种解释，表明模型会同时赋予两种可能性权重。最后，语言模型不会复用花园路径句处理中的特征来回答后续问题。
原文摘要 · Abstract (English)
Autoregressive transformer language models (LMs) possess strong syntactic abilities, often successfully handling phenomena from agreement to NPI licensing. However, the features they use to incrementally process language inputs are not well understood. In this paper, we fill this gap by studying the mechanisms underlying garden path sentence processing in LMs. We ask: (1) Do LMs use syntactic features or shallow heuristics to perform incremental sentence processing? (2) Do LMs represent only one potential interpretation, or multiple? and (3) Do LMs reanalyze or repair their initial incorrect representations? To address these questions, we use sparse autoencoders to identify interpretable features that determine which continuation - and thus which reading - of a garden path sentence the LM prefers. We find that while many important features relate to syntactic structure, some reflect syntactically irrelevant heuristics. Moreover, while most active features correspond to one reading of the sentence, some features correspond to the other, suggesting that LMs assign weight to both possibilities simultaneously. Finally, LMs do not re-use features from garden path sentence processing to answer follow-up questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。