探究变换器如何编码语音特征，揭示其跨语言泛化局限
Probing Subphonemes in Morphology Models
- 设计无语言依赖的探针方法，分析音素嵌入中的语音特征
- 发现局部特征如土耳其语词尾浊音清化被音素嵌入准确捕捉
- 长距离依赖如元音和谐更依赖编码器表示，对模型训练有启示
Transformer 在形态变化任务中表现卓越，但跨语言和形态规则的泛化能力仍受限。一个可能原因是模型对语音及亚音素层面隐含现象的捕捉程度。本文提出一种无语言依赖的探针方法，研究在直接以音素训练的 Transformer 中语音特征的编码情况，并在七种形态多样的语言上进行实验。结果表明，局部语音特征（如土耳其语词尾浊音清化）在音素嵌入中表达良好，而长距离依赖（如元音和谐）则更优地体现在 Transformer 编码器中。这些发现为形态模型的训练策略提供了实证依据，尤其强调了亚音素特征习得的重要性。
原文摘要 · Abstract (English)
Transformers have achieved state-of-the-art performance in morphological inflection tasks, yet their ability to generalize across languages and morphological rules remains limited. One possible explanation for this behavior can be the degree to which these models are able to capture implicit phenomena at the phonological and subphonemic levels. We introduce a language-agnostic probing method to investigate phonological feature encoding in transformers trained directly on phonemes, and perform it across seven morphologically diverse languages. We show that phonological features which are local, such as final-obstruent devoicing in Turkish, are captured well in phoneme embeddings, whereas long-distance dependencies like vowel harmony are better represented in the transformer's encoder. Finally, we discuss how these findings inform empirical strategies for training morphological models, particularly regarding the role of subphonemic feature acquisition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。