提出神经符号混合模型,统一语法的层级结构与线性预测过程。
Shadow of the (Hierarchical) Tree: Reconciling Symbolic and Predictive Components of the Neural Code for Syntax
- 用ROSE架构区分高层层级表示与低层统计推断
- 预测编码机制连接符号相位码与神经群体码
- 适合研究语言神经机制与人工模型的交叉应用
自然语言语法是整合符号表征与联结主义神经网络两大框架的重要测试场。基于近期语法的神经计算架构ROSE,本文探讨了如何通过混合神经符号模型调和神经代码中层次化的‘垂直’语法与线性、预测性的‘水平’过程。认为高层ROSE可表征垂直短语结构,而低层则通过统计与感知推断实现水平语言信息建模。预测表明,人工语言模型对水平形态句法的认知神经科学研究贡献较大,但对层级组合结构影响有限。该视角有助于解决当前文献中的诸多矛盾。讨论了两类神经代码的融合路径,特别强调预测编码可作为符号振荡相位码与线性化语法统计群体码之间的接口。最后,提出一个数学模型,实现将符号表征注入编码词义统计特征的神经系统。
原文摘要 · Abstract (English)
Natural language syntax can serve as a major test for how to integrate two infamously distinct frameworks: symbolic representations and connectionist neural networks. Building on a recent neurocomputational architecture for syntax (ROSE), I discuss the prospects of reconciling the neural code for hierarchical 'vertical' syntax with linear and predictive 'horizontal' processes via a hybrid neurosymbolic model. I argue that the former can be accounted for via the higher levels of ROSE in terms of vertical phrase structure representations, while the latter can explain horizontal forms of linguistic information via the tuning of the lower levels to statistical and perceptual inferences. One prediction of this is that artificial language models will contribute to the cognitive neuroscience of horizontal morphosyntax, but much less so to hierarchically compositional structures. I claim that this perspective helps resolve many current tensions in the literature. Options for integrating these two neural codes are discussed, with particular emphasis on how predictive coding mechanisms can serve as interfaces between symbolic oscillatory phase codes and population codes for the statistics of linearized aspects of syntax. Lastly, I provide a neurosymbolic mathematical model for how to inject symbolic representations into a neural regime encoding lexico-semantic statistical features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。