arXiv:2609.04484cs.CLcs.AI2026-09

语言模型生成时会受前文结构影响,且语义越连贯,影响越明显。

Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation

论文配图:Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation
图 1 · 摘自论文原文
  • 通过控制句式实验,研究模型生成时的结构预示效应。
  • 高频的介词宾语结构绝对提升更大,但双宾结构相对增幅更显著。
  • 语义连贯性增强结构预示,且生成内容重复度更高。

本文研究语言模型在生成过程中结构预示现象,考察前文结构如何影响句子续写。尽管已有研究证明结构交替在理解中存在预示效应,但其在生成阶段的表现尚不明确,因生成时模型每一步都从多个可能延续中采样。我们通过一系列针对双宾结构的控制式句式补全实验进行探究。结果表明,语言模型确实存在结构预示,尤其在语义连贯的句子中更为明显。在预示强度上,虽然双宾结构相对于基线有更大的相对增幅,符合逆频率效应;但介词宾语结构(更常使用)则表现出更大的绝对增幅。此外,不仅语义一致性能增强结构预示,且结构预示后的生成内容也展现出更高的词汇与语义重复水平。综合来看,证据支持语言模型中的结构预示作用跨越句法、词汇和语义多个层次,促进并受这些层次对齐的共同影响。

原文摘要 · Abstract (English)

This paper investigates structural priming in language model (LM) production, examining how preceding structural context influences sentence completion. While prior work has demonstrated priming effects in comprehension of structural alternations, it remained unclear whether these persist in production, where, when generating, an LM samples from many possible continuations at each step. We address this question through a series of controlled sentence-completion experiments on dative constructions. In line with prior work, we find that LMs are susceptible to structural priming, particularly in sentences that are semantically coherent. In terms of priming magnitude, we find that while there is a greater relative increase of double-object datives against our baselines, in line with inverse frequency effects, there is a larger absolute increase in prepositional-objects, the more frequently produced construction. Finally, we not only observe that structural priming is boosted by lexico-semantic coherence, but that structurally primed completions display greater levels of lexico-semantic repetition. Taken together, our evidence supports the view that structural priming in LMs operates across multiple levels of linguistic representation, facilitating, and facilitated by syntactic, lexical, and semantic alignment. Code: https://github.com/the-context-lab/primedproduction.

语言模型结构预示生成机制语义连贯

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。