LLMs捕捉到UD树库未涵盖的句法结构,支持最简方案相位理论。
Probing LLMs for Syntactic Structure Beyond Universal Dependencies: A Minimalist Phase Account in English
- 用嵌入子句的主语-动词距离探测句法差异,发现有限式与不定式结果相反。
- 13个模型中多数层在有限式下距离更短,不定式下更长,符号反转无法用传统理论解释。
- 仅靠动词替换即可反转结果,证明是深层句法结构而非表面线索。
我们发现,大型语言模型编码了通用依存关系(UD)树距离无法反映的句法区别。在英语疑问词移动刺激下,嵌入子句的主语与其动词间的探测距离在不同条件下保持不变(基于UD树距离)。然而,当嵌入子句为有限式时,该距离短于基线;若为不定式,则更长——这一子句内符号不对称性在13个模型的四个家族中均出现,且遍布多数层。任何基于UD距离、线性顺序或单调结构复杂性的理论都无法产生符号反转,而最简方案相位理论可以。固定主句动词,仅改变补足语类型,仍可复现相同有限式-不定式排序,排除了词汇语义解释。跨子句对也重现了先前研究中的相位计数排序,验证了探测器的有效性。一次激活修补操作,交换选择子句的主句动词,使两组对在8/13个模型中反向移动。由于目标字符串不变,该因果干预排除了子句长度和表面线索的解释。这些结果共同揭示了超出UD探测目标的模型句法结构,且最简方案相位理论与此一致。
原文摘要 · Abstract (English)
We show that LLMs encode syntactic distinctions not present in the Universal Dependencies (UD) tree distances that structural probes are trained to recover. On English wh-movement stimuli, we measure the probe distance between an embedded subject and its verb, whose UD tree distance is invariant across conditions. That distance is shorter than baseline when the embedded clause is finite and longer when it is infinitival -- a within-clause sign asymmetry present in all 13 models across four families we test, at a majority of layers. No account based on UD distance, linear order, or monotone structural complexity can produce a sign reversal, while Minimalist phase theory can. Holding the matrix verb fixed while varying only the complement type reproduces the same finite-infinitival ordering in every model, ruling out a lexical-semantic explanation. The cross-clause pair separately reproduces the phase-count ordering of earlier work, validating the probes. A single activation patch, interchanging the clause-selecting matrix verb, moves the two pairs in opposite directions in 8 of 13 models. Because the causal intervention leaves the target string unchanged, it also rules out the clause-length and surface-cue explanations. Together these results reveal syntactic structure in LLMs beyond the UD probe target, and a Minimalist phase account is consistent with it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。