arXiv:2410.21013cs.CL2024-10ACL被引 4

研究变压器模型如何学习西班牙语动词不规则形态,发现频率影响记忆与规则化。

Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers

  • 用输入频率调控实验,测试模型对不规则形态的习得机制。
  • 在频次不均时,模型对L型不规则动词表现更好,且越常出现越易记忆。
  • 罕见音变导致模型倾向将不规则词变成规则词,适合语言模型研究者参考。

过去十年间,多项研究探讨了不同语言中说话者如何解决所谓的‘词形表填空问题’(PCFP)。该问题关注的是:当词形表不完整时,说话者如何准确生成词语的屈折形式?这一问题在复杂屈折系统中尤为突出。本文聚焦于西班牙语动词形态,其中某些动词遵循不规则的L形模式——第一人称单数现在时词干与现在虚拟语气中使用的一致。我们将此问题建模为形态重赋格任务,重点研究输入频率在模型习得规则与不规则L形模式中的作用。通过系统性地调节输入分布并分析模型行为,得出四个关键发现:1)模型在不规则的L形动词上表现优于规则动词,尤其在频次不均条件下;2)存在显著的首因效应,但无一致的近因效应;3)随着L形动词比例增加,模型更倾向于记忆而非推导;4)当训练数据中缺乏特定辅音交替对时,模型有将不规则动词规则化的倾向。

原文摘要 · Abstract (English)

Over the past decade, various studies have addressed how speakers solve the so-called `The Paradigm Cell Filling Problem' (PCFP) \citep{ackerman2009parts} across different languages. The PCFP addresses a fundamental question in morphological processing: how do speakers accurately generate inflected forms of words when presented with incomplete paradigms? This problem is particularly salient when modeling complex inflectional systems. We focus on Spanish verbal paradigms, where certain verbs follow an irregular L-shaped pattern, where the first-person singular present indicative stem matches the stem used throughout the present subjunctive mood. We formulate the problem as a morphological reinflection task. Specifically, we investigate the role of input frequency in the acquisition of regular versus irregular L-shaped patterns in transformer models. By systematically manipulating the input distributions and analyzing model behavior, we reveal four key findings: 1) Models perform better on L-shaped verbs compared to regular verbs, especially in uneven frequency conditions; 2) Robust primacy effects are observed, but no consistent recency effects; 3) Memorization becomes more prominent as the proportion of L-shaped verbs increases; 4) There is a tendency to regularize L-shaped verbs when their consonant alternation pairs are rare or absent in the training data.

形态学Transformer语言模型西班牙语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。