揭示低秩RNN学习中隐藏的连接结构，解释训练如何暴露功能等价网络差异。
Learning reveals invisible structure in low-rank RNNs
- 将梯度下降映射到低维重叠空间，建立可解析的学习动力学方程。
- 发现不影响输出的隐藏重叠项能记录训练历史，形成记忆变量。
- 理论可指导生物学习实验设计，预测可验证现象。
神经系统的学习源于突触变化对行为表征的重塑。尽管低秩循环神经网络（RNN）已成为连接结构与功能的重要框架，其学习过程的理论理解仍不清晰。本文将低秩框架从活动扩展至学习，直接在简化重叠空间中推导梯度下降动力学。我们提出一个封闭形式的低维常微分方程组，精确描述线性RNN的学习过程，并在大N高斯极限下对非线性RNN渐近精确。分析核心在于区分两类重叠：损失可见重叠（完全决定网络活动、输出和损失），以及损失不可见重叠（不影响功能但必须用于描述学习）。通过两个现象阐明此分解的后果：第一，学习可作为扰动，揭示功能等价网络间的连接差异；第二，损失不可见重叠可充当记忆变量，编码训练历史，并给出其发生条件。最后，基于理论提出多个可验证的生物学学习实验预测。
原文摘要 · Abstract (English)
Learning in neural systems arises from synaptic changes that reshape the representations underlying behavior. While low-rank recurrent neural networks (RNNs) have emerged as a powerful framework for linking connectivity to function, a theoretical understanding of their learning process remains elusive. Here, we extend the low-rank framework from activity to learning by deriving gradient-descent dynamics directly in a reduced overlap space. We formulate a closed-form, low-dimensional system of ODEs that governs learning in this space, exact for linear RNNs and asymptotically exact for nonlinear RNNs in the large-$N$ Gaussian limit. Central to our analysis is a distinction between two classes of overlaps: loss-visible overlaps, which fully determine network activity, output, and loss, and loss-invisible overlaps, which do not affect function but are required to describe learning. We illustrate the consequences of this decomposition through two phenomena. First, we show that learning can serve as a perturbation that exposes differences in connectivity between functionally equivalent networks. Second, we show that loss-invisible overlaps can act as memory variables that encode training history, and characterize the conditions under which this occurs. Finally, we present several testable predictions for biological learning experiments derived from our theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。