受冯·诺依曼脑模型启发,构建可自组织的高效神经网络。
Von Neumann Networks

- 基于细胞阵列的可学习神经元,角色由扩散过程决定。
- 多层网络在基础任务上性能更优,参数量减少30%以上。
- 适合研究计算架构演化与低资源神经网络设计者。
20世纪中期,数学家冯·诺依曼提出一种基于细胞阵列的计算系统,模拟人脑结构,每个细胞具有有限状态并遵循扩散过程。本文将该思想融入现代深度学习,构建了可学习的专用角色神经元,称为冯·诺依曼神经元(Von Neumann Neuron),由此形成的神经网络(VNN)具有自工程化架构,其结构仅依赖于输入输出在细胞阵列中的位置。理论框架表明,这类网络基于神经算子扩展,通过在具有扩散特性的细胞拓扑上学习格林函数与卷积。我们证明,这些网络属于更广泛的计算系统——细胞机(Cellular Machines),具备图灵完备性。初步实验显示,基于VNN的多层感知机在基础任务中优于等效深度学习模型,参数效率更高,并能学习新任务。该方法还可用于构造扩展的冯·诺依曼(硬件)架构,为新型计算范式提供可能。
原文摘要 · Abstract (English)
In the mid-twentieth century, mathematician and polymath John von Neumann created a computational system on an array of cells as a simple model of the human brain, where each cell had one of a finite set of roles or states that he predicted would be modelled by a diffusion process. In this work, we show that such a system, when developed in a modern deep learning setting, enables the construction of an artificial neuron having specialized roles that can be learnt. We refer to this neuron as the Von Neumann neuron, and the resulting neural network from such neurons result in a self-engineered design whose architecture is only dependent on the structure and locations of its inputs and outputs on this cellular array. The mathematical framework for these Von Neumann Networks (VNNs) is also constructed and shows that they are based on the extension of neural operators and the learning of Green's functions with convolutions on a cellular topology having a diffusion signature. We also prove that these VNNs are part of a more general computational system called Cellular Machines that are computationally universal. Initial experiments show that VNN based multi-layered perceptrons outperform their equivalent deep learning variant on basic tasks, while being more parameter efficient and are capable of learning new types of tasks. This includes the ability to solve for and construct an extension of the Von Neumann (hardware) architecture common to all modern computers to cells and suggests new opportunities that could be explored.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。