arXiv:2505.09005cs.CL2025-05被引 2

GPT-4能像人类一样预测长距离依赖句的可接受性,基于信息结构。

For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependencies

  • 用零样本任务测试GPT-4对信息结构与句法可接受性的理解
  • 发现其判断结果与人类高度一致,且具因果关系
  • 适合语言学、NLP可解释性研究者阅读

目前仍存在争议:大语言模型(LM)是否真正理解自然语言,或能否生成可靠的元语言判断。此外,较少研究证明LM能够表征并尊重语言学家提出的细微形式-功能关系。本文聚焦近期研究确立的一种关系:英语母语者对标准句的信息结构判断,可预测独立收集的长距离依赖(LDD)构造的可接受性评分,涵盖多种基础句型和多种类型的LDD。为检验任一LM是否捕捉此关系,我们对GPT-4执行与人类相同的任务及新扩展任务。结果显示,GPT-4在信息结构和可接受性任务中表现出可靠元语言能力,复现了二者间的显著交互作用,尽管任务为零样本且显式,几乎无污染可能(研究1a, 1b)。研究2通过操纵基础句的信息结构,证实:提升上下文中某一成分的突出性,会提高后续对LDD构造的可接受性评分。这些发现表明,自然语言与GPT-4生成的英语之间存在紧密关联,信息结构与句法之间亦然,亟待进一步探索。

原文摘要 · Abstract (English)

It remains debated how well any LM understands natural language or generates reliable metalinguistic judgments. Moreover, relatively little work has demonstrated that LMs can represent and respect subtle relationships between form and function proposed by linguists. We here focus on a particular such relationship established in recent work: English speakers' judgments about the information structure of canonical sentences predicts independently collected acceptability ratings on corresponding 'long distance dependency' [LDD] constructions, across a wide array of base constructions and multiple types of LDDs. To determine whether any LM captures this relationship, we probe GPT-4 on the same tasks used with humans and new extensions.Results reveal reliable metalinguistic skill on the information structure and acceptability tasks, replicating a striking interaction between the two, despite the zero-shot, explicit nature of the tasks, and little to no chance of contamination [Studies 1a, 1b]. Study 2 manipulates the information structure of base sentences and confirms a causal relationship: increasing the prominence of a constituent in a context sentence increases the subsequent acceptability ratings on an LDD construction. The findings suggest a tight relationship between natural and GPT-4 generated English, and between information structure and syntax, which begs for further exploration.

语言模型信息结构句法可接受性认知模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。