对比大模型、指令模型与人类对重复词语的处理差异,发现训练方式导致行为本质不同。
Automatic or Controlled? Repetition Priming Reveals Divergent Processing in Base LLMs, Instruct LLMs, and Humans

- 用重复启动实验测试15个模型,比较其对重复词的反应机制。
- 基础模型自动处理,指令模型需主动调控,人类介于二者之间。
- 模型规模越大,指令模型越偏离自动处理,揭示后训练的影响。
自然语言中词汇频繁重复,但语言模型是否重用先前表征或重新评估尚不明确,且后训练是否改变这一行为仍未知。我们对15个模型(参数量1.5B–14B)在五个模型家族中,采用重复启动范式(Shiffrin and Schneider, 1977),在语义分类和填空任务中进行测试,并与人类使用相同刺激物的实验进行对照。结果发现:基础模型表现出自动加工特征——即时促进效应稳定存在于不同滞后条件下,部分抵抗上下文移除,且与先前出现的关注度相关;指令模型则呈现受控加工——促进效应随滞后增加而衰减,无预期上下文时完全消失,且在更大滞后下转为干扰;在Qwen 2.5系列中,该差异随模型规模单调上升,表明后训练逐步改变重复信息处理方式。人类表现出混合模式,具有滞后敏感的促进效应(类似指令模型),但无干扰现象,说明两种模型均未完全模拟人类认知。研究揭示了后训练引发的语言模型重复处理行为的根本转变,提供了行为机制层面的实证证据。
原文摘要 · Abstract (English)
Words recur constantly in natural language use, yet it remains unclear whether language models reactivate prior representations or re-evaluate repeated words afresh, and whether post-training changes this default behavior. We apply repetition priming (Shiffrin and Schneider, 1977) to 15 models across five model families (1.5B-14B parameters) in two tasks, semantic categorization and cloze completion, with matched human experiments using identical stimuli. We find that base models exhibit automatic processing: they show immediate facilitation that remains stable across lags, partially survives context removal, and correlates with attention to prior occurrences. Instruct models exhibit controlled processing: their facilitation decays with lag, collapses without expected context, and reverses to interference at larger scales. Within the Qwen 2.5 family, this dissociation increases monotonically with model scale, suggesting that post-training progressively alters repetition processing. Humans show a hybrid profile, with lag-sensitive facilitation resembling instruct models but without interference, suggesting that neither model type fully captures human cognition. Our findings reveal a qualitative shift in how language models process repeated information after post-training and provide mechanistic evidence for the divergence between model behaviors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。