用深度网络替代线性映射,提升语言理解模型精度。
Is deeper always better? Replacing linear mappings with deep learning networks in the Discriminative Lexicon Model
- 用深层神经网络替换线性映射,构建新语言模型
- 在英荷语大数据上表现更优,但对爱沙尼亚语和台湾普通话效果不佳
- 适合研究词形结构复杂语言的学者,尤其关注增量学习场景
深度学习模型在语言认知建模中应用日益广泛。本文探讨深度学习能否在语音与语义向量映射上超越传统线性方法。基于Baayen提出的判别词库模型(Discriminative Lexicon Model),将原有线性映射(LDL)替换为深层密集神经网络(DDL)。结果显示,DDL在英语和荷兰语的大规模多样化数据集上表现更优,但在爱沙尼亚语和台湾普通话上未见优势。对于具有伪构词结构的词(如chol+er),DDL显著优于LDL。在反应时预测任务中,频率感知的线性模型(FIL)仍优于普通DDL;但频率感知的深度学习模型(FIDDL)则大幅超越FIL。此外,线性映射能有效实现试次间更新以模拟词汇增量学习,而深度映射则难以做到。当前线性和深度映射均有助于理解语言机制。
原文摘要 · Abstract (English)
Recently, deep learning models have increasingly been used in cognitive modelling of language. This study asks whether deep learning can help us to better understand the learning problem that needs to be solved by speakers, above and beyond linear methods. We utilise the Discriminative Lexicon Model introduced by Baayen and colleagues, which models comprehension and production with mappings between numeric form and meaning vectors. While so far, these mappings have been linear (Linear Discriminative Learning, LDL), in the present study we replace them with deep dense neural networks (Deep Discriminative Learning, DDL). We find that DDL affords more accurate mappings for large and diverse datasets from English and Dutch, but not necessarily for Estonian and Taiwan Mandarin. DDL outperforms LDL in particular for words with pseudo-morphological structure such as chol+er. Applied to average reaction times, we find that DDL is outperformed by frequency-informed linear mappings (FIL). However, DDL trained in a frequency-informed way ('frequency-informed' deep learning, FIDDL) substantially outperforms FIL. Finally, while linear mappings can very effectively be updated from trial-to-trial to model incremental lexical learning, deep mappings cannot do so as effectively. At present, both linear and deep mappings are informative for understanding language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。