arXiv:2506.06134q-bio.NCcs.LG2025-06被引 1

提出一种多时标神经网络,实现主子空间投影的稳定收敛。

Similarity Matching Networks: Hebbian Learning and Convergence Over Multiple Time Scales

  • 设计三时标耦合网络:快(神经动力)、中(侧向突触)、慢(前馈突触)
  • 证明三阶段动态分别收敛:快速全局指数收敛、中速在正定矩阵空间收敛、慢速几乎必然收敛到全局最优
  • 结合赫布与反赫布学习规则,具生物学合理性,适合神经计算研究者

近期在生物合理性的降维规范框架中,基于相似性匹配代价函数和低秩矩阵逼近问题取得突破。尽管已有明确生物学解释、跨领域成功应用及实验验证,但完整的收敛分析仍不明确。本文构建并分析一个连续时间神经网络——相似性匹配网络,用于主子空间投影。该网络源自极小极大极小目标,包含三个在不同时间尺度上耦合的动力学:神经动力(快)、侧向突触动力(中)、前馈突触动力(慢)。前馈与侧向突触动力分别由赫布和反赫布学习规则构成。通过多层级优化框架,我们在离线设置下证明了动态的收敛性:第一层(快时标)中,代价函数强凸,梯度流动态全局指数收敛;第二层(中时标)中,代价函数强凹,梯度流动态在正定矩阵空间内指数收敛;第三层(慢时标)中,研究非凸非光滑代价函数,给出其全局最小值的显式表达,并证明梯度流动态几乎必然收敛至全局最小值。这些结果依赖两个经数值实验充分支持的经验假设。最后,通过数值实例验证了方法的有效性。

原文摘要 · Abstract (English)

A recent breakthrough in biologically-plausible normative frameworks for dimensionality reduction is based upon the similarity matching cost function and the low-rank matrix approximation problem. Despite clear biological interpretation, successful application in several domains, and experimental validation, a formal complete convergence analysis remains elusive. Building on this framework, we consider and analyze a continuous-time neural network, the \emph{similarity matching network}, for principal subspace projection. Derived from a min-max-min objective, this biologically-plausible network consists of three coupled dynamics evolving at different time scales: neural dynamics, lateral synaptic dynamics, and feedforward synaptic dynamics at the fast, intermediate, and slow time scales, respectively. The feedforward and lateral synaptic dynamics consist of Hebbian and anti-Hebbian learning rules, respectively. By leveraging a multilevel optimization framework, we prove convergence of the dynamics in the offline setting. Specifically, at the first level (fast time scale), we show strong convexity of the cost function and global exponential convergence of the corresponding gradient-flow dynamics. At the second level (intermediate time scale), we prove strong concavity of the cost function and exponential convergence of the corresponding gradient-flow dynamics within the space of positive definite matrices. At the third and final level (slow time scale), we study a non-convex and non-smooth cost function, provide explicit expressions for its global minima, and prove almost sure convergence of the corresponding gradient-flow dynamics to the global minima. These results rely on two empirically motivated conjectures that are supported by thorough numerical experiments. Finally, we validate the effectiveness of our approach via a numerical example.

神经网络收敛分析赫布学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。