语言模型如何避免过度泛化？研究发现它们依赖抽象层面的间接证据，而非具体动词的排斥机制。
Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralization
- 通过控制训练数据中近义结构的出现，区分了排斥与固化两种机制
- 模型在动词层面未表现出排斥效应，但在抽象层面有微弱但显著的抑制作用
- 结果提示神经网络需具备对间接负向证据的敏感性，适合认知科学与语言学习研究者
学习者如何避免过度泛化（如 Tom laughed me）而无需直接否定反馈？建构主义提出两种间接负向证据机制：排斥（优先接触近义结构，如 she made him laugh）与固化（所有动词的语法使用，包括 He laughed）。本研究通过在儿童-照护者对话数据上训练的语言模型进行受控培育实验，系统性地移除排斥性与非排斥性证据。结果表明，尽管模型能避免过度泛化，但在动词层面并未显示排斥效应，仅在抽象层面存在微弱但非零的排斥迹象。结合训练动态分析发现，在动词特定条件下，模型将竞争结构视为间接正向证据而非负向证据。若排斥是人类避免过度泛化的更合理路径，则该结果提示神经网络学习者需具备对间接负向证据的敏感性，并建议开展新的人类实验验证抽象排斥机制。
原文摘要 · Abstract (English)
How do learners avoid overgeneralizations such as Tom laughed me without explicit negative evidence? Constructionists have posited two proposals that describe indirect negative evidence against overgeneralizations: preemption (which privileges exposure to near-synonymous construction---e.g., she made him laugh) vs. entrenchment (all exposures to a verb's grammatical usages, including cases like He laughed). We disentangle these hypotheses by running controlled rearing experiments on LMs trained on child-caregiver conversations, where we systematically remove preemptive vs. non-preemptive evidence. We find that while LMs avoid overgeneralizations, they do not show preemption at a verb-specific level, instead showing weak but non-zero evidence of abstract preemption. Combined with results from analyzing the LMs' training dynamics, we find that LMs treat competing structures as indirect positive---as opposed to negative---evidence in the verb-specific condition. Insofar as preemption is the more plausible route to avoiding overgeneralizations in humans, our results point the need for there to be sensitivities to indirect negative evidence in neural network learners, and suggest new human experiments to test abstract preemption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。