模型学会西班牙语特殊变位模式,但仅记住了具体单词而非规则。
Probing Character-level Transformers for the Spanish L-shaped Morphome
- 通过探针分析发现模型编码了特定词类,而非表面变化规律。
- 在无变化实例中仍能识别该词类,且编码位置在解码器中间层。
- 学习的单词个体比模型结构更重要,无法像人一样推广规则。
当变换器学习不规则形态模式时,它究竟学到了什么?以西班牙语的‘L形词形’为例,该模式在第一人称单数陈述式和所有虚拟语气形式中,动词词干发生精确交替,且无语音、语义或句法特征可预测其归属。先前研究显示字符级变换器能复现此模式,但仅描述输出结果,未揭示内在表征。我们对五种架构、每种十二个训练模型进行测试,在词干不重叠交叉验证下设置控制组与表面基线。结果表明:模型编码的是L形词类本身,而不仅是可见的词干交替;其编码在所有表面基线之上可解码,即使所有形式词干相同也有效;在交替实例上训练的探针仍可分类非交替实例。编码位置位于解码器中间层的词干末辅音位置,即词干选择阶段之前。且编码具有词项特异性:模型学到的具体动词比其架构更重要。模型将该词形模式作为词项特定的词汇抽象存储,足以复现模式,但无法像人类一样泛化。
原文摘要 · Abstract (English)
When a transformer learns an irregular morphological pattern, what has it learned? Our test case is the Spanish \emph{L-shaped morphome}, a complex irregular pattern in which the verb's stem alternates in exactly the first-person singular indicative and all subjunctive forms, and whose membership no phonological, semantic, or syntactic feature predicts. Prior studies have shown that character-level transformers can reproduce this pattern, but that evidence describes what models produce, not what they represent. Probing five architectures, twelve trained models each, under lemma-disjoint cross-validation with controls and surface baselines, we show that the models encode the L-shaped class itself, not just its visible alternations. It is decodable above every surface baseline, survives instances in which every form shows the same stem, and probes trained on alternating instances still classify non-alternating ones. The encoding is localized where the stem choice is made, at the stem-final consonant position of the middle decoder, before the alternant is read. And it is item-specific: which verbs a model learned matters far more than which architecture it is. The models store the morphome as an item-specific lexical abstraction, sufficient to reproduce the pattern but not to generalize it as humans do.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。