arXiv:2605.03598cs.NEcs.AI2026-05

用图论方法揭示神经网络如何通过多跳路径实现计算。

Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks

论文配图:Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks
图 1 · 摘自论文原文
  • 将神经网络视为图,分析输入输出间的多跳路径
  • 新模型R-RNN在稀疏任务下性能更优,时间稀疏性更强
  • 适合研究神经网络结构与功能关系的学者

理解生物与人工神经网络如何从连接结构实现计算,是神经科学和机器学习的核心问题。在神经系统中,结构连接与功能连接常不一致,促使研究超越直接连接。本文发现,训练于分层模块化任务的循环神经网络(RNN)的空间与时间功能,可通过将网络建模为图并分析输入输出间的多跳路径来恢复。分解不同跳数的路径可揭示信息的时间路由机制。这一视角重新定义正则化:若功能依赖多跳通信,则传统L1正则化仅约束单跳结构,而非支撑计算的多跳路径。为此,我们提出残差-RNN(R-RNN),通过约束多跳路径,实现超越标准L1正则化的时序稀疏性。相比L1,R-RNN在稀疏任务信号下表现更优,且具备更强的稀疏-功能对齐,在强正则化下更具鲁棒性。结果表明,多跳通信是连接循环网络结构与功能的关键机制,稀疏性应基于功能路径而非单个参数定义。

原文摘要 · Abstract (English)

Understanding how biological and artificial neural networks implement computation from connectivity is a central problem in neuroscience and machine learning. In neural systems, structural and functional connectivity are known to diverge, motivating approaches that move beyond direct connections alone. Here, we show that the spatial and temporal function of recurrent neural networks (RNNs) trained on hierarchically modular tasks can be recovered by modelling the network as a graph and analysing the multi-hop pathways between input and output units. In particular, decomposing these pathways by hop length reveals how the network temporally routes information. This perspective reframes regularisation: if function is implemented through multi-hop communication, then standard penalties such as L1 regularisation, which act only on individual weights, constrain single-hop structure rather than the multi-hop pathways that support computation. Motivated by this view, we introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and thereby induce temporal sparsity beyond that achieved by standard L1 regularisation. Compared with L1 regularisation, R-RNNs achieve improved performance by inducing temporal sparsity that matches the task structure, even when the task signal is sparse. Moreover, R-RNNs exhibit stronger sparsity-function alignment, reflected in their increased robustness under strong regularisation. Together, our results identify multi-hop communication as a key principle linking structure to function in recurrent networks, and suggest that sparsity should be defined over functional pathways rather than individual parameters.

神经网络图论稀疏性动态系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。