揭示循环图神经网络与实数算术电路的等价性,为理解模型能力提供新视角。
Recurrent Graph Neural Networks and Arithmetic Circuits
- 提出循环算术电路模型,用记忆门实现迭代计算
- 证明循环图神经网络与实数算术电路可完全相互模拟
- 为研究神经网络提供电路复杂性理论的新工具
我们从实数上的算术电路角度刻画了循环图神经网络(GNN)的计算能力。所提网络不限于聚合-组合型GNN或其他特定类型。通过推广文献中的类似概念,引入循环算术电路模型,其可视为序列或逻辑电路的算术类比。该电路利用记忆门在迭代间存储数据。尽管(循环)GNN作用于带标签图,我们构建的算术电路将编码后的标签图作为实数元组输入,并计算相同函数。反向则构造能模拟循环电路计算的循环GNN:将电路输入作为初始特征向量,经GNN计算后,电路输出即包含在节点特征中。由此建立了循环GNN与实数上运行的循环算术电路之间的精确对应关系。研究成果深化了对训练后神经网络能力的理解,并开辟了以电路复杂性理论视角研究循环神经网络的新路径。
原文摘要 · Abstract (English)
We characterise the computational power of recurrent graph neural networks (GNNs) in terms of arithmetic circuits over the real numbers. Our networks are not restricted to aggregate-combine GNNs or other particular types. Generalising similar notions from the literature, we introduce the model of recurrent arithmetic circuits, which can be seen as arithmetic analogues of sequential or logical circuits. These circuits utilise so-called memory gates which are used to store data between iterations of the recurrent circuit. While (recurrent) GNNs work on labelled graphs, we construct arithmetic circuits that obtain encoded labelled graphs as real valued tuples and then compute the same function. For the other direction we construct recurrent GNNs which are able to simulate the computations of recurrent circuits. These GNNs are given the circuit-input as initial feature vectors and then, after the GNN-computation, have the circuit-output among the feature vectors of its nodes. In this way we establish an exact correspondence between the expressivity of recurrent GNNs and recurrent arithmetic circuits operating over real numbers. Our results both deepen our understanding of the capabilities of trained neural networks and open new approaches to study recurrent neural networks using the lens of circuit complexity theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。