用大模型研究人类极性错觉，发现大小模型表现不同
What can LLMs tell us about the mechanisms behind polarity illusions in humans? Experiments across model scales and training steps
- 用Pythia系列模型测试极性错觉的生成机制
- 大模型中否定词错觉减弱，深度炸弹错觉增强
- 支持浅层处理理论，适合语言认知研究者
本文使用Pythia规模系列（Biderman et al. 2023）研究大型语言模型中两种经典的极性错觉——否定词错觉（NPI illusion）和深度炸弹错觉（depth charge illusion）是否出现及其演化规律。结果表明，随着模型规模增大，否定词错觉逐渐减弱并最终消失，而深度炸弹错觉则在更大模型中变得更明显。这一现象暗示：解释人类极性错觉未必需要假设存在“理性推理”机制将病句转化为正句，因为大模型无法进行此类隐含层面的推理。相反，模型中可能仅存在浅层、足够好的处理方式，或对规范上不合法结构的部分语法化。本文提出一种基于构式语法基本原理的理论整合方案。
原文摘要 · Abstract (English)
I use the Pythia scaling suite (Biderman et al. 2023) to investigate if and how two well-known polarity illusions, the NPI illusion and the depth charge illusion, arise in LLMs. The NPI illusion becomes weaker and ultimately disappears as model size increases, while the depth charge illusion becomes stronger in larger models. The results have implications for human sentence processing: it may not be necessary to assume "rational inference" mechanisms that convert ill-formed sentences into well-formed ones to explain polarity illusions, given that LLMs cannot plausibly engage in this kind of reasoning, especially at the implicit level of next-token prediction. On the other hand, shallow, "good enough" processing and/or partial grammaticalization of prescriptively ungrammatical structures may both occur in LLMs. I propose a synthesis of different theoretical accounts that is rooted in the basic tenets of construction grammar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。