通过局部动态机制实现无梯度学习,让神经网络自动找到高效表征。
Dynamical Learning in Deep Asymmetric Recurrent Neural Networks
- 利用非对称连接和自耦合,生成指数级大的稳定配置流形。
- 无需反向传播,在标准图像分类上性能媲美多层感知机。
- 适合生物启发与类脑计算架构,突破传统误差驱动学习范式。
我们研究具有非对称连接的递归神经网络,发现引入自耦合或稀疏兴奋性模块间连接,可催生一个密集连通的动态可达稳定配置流形。该流形规模随系统尺寸呈指数增长,且可通过简单局部动力学到达,尽管它仅占全局配置空间的子主导部分。我们进一步证明,可直接在该结构上实施完全局部、无梯度的学习机制,选择性地稳定单个任务相关网络配置。与误差驱动或对比学习不同,该方法无需显式比较有无输出监督时的网络状态。瞬时监督信号将动力学引导至表示流形后,局部可塑性固化所达配置,从而有效塑造潜在表征空间。在标准图像分类基准上的数值评估表明,其性能可与使用反向传播训练的多层感知机相媲美。这些结果表明,固定点的动力学可达性以及内部动力学的稳定化,为递归系统的学习提供了可行替代原则,与统计物理有概念关联,并可能对生物启发及神经形态计算架构产生影响。
原文摘要 · Abstract (English)
We investigate recurrent neural networks with asymmetric interactions and demonstrate that the inclusion of self-couplings or sparse excitatory inter-module connections leads to the emergence of a densely connected manifold of dynamically accessible stable configurations. This representation manifold is exponentially large in system size and is reachable through simple local dynamics, despite constituting a subdominant subset of the global configuration space. We further show that learning can be implemented directly on this structure via a fully local, gradient-free mechanism that selectively stabilizes a single task-relevant network configuration. Unlike error-driven or contrastive learning schemes, this approach does not require explicit comparisons between network states obtained with and without output supervision. Instead, transient supervisory signals bias the dynamics toward the representation manifold, after which local plasticity consolidates the attained configuration, effectively shaping the latent representation space. Numerical evaluations on standard image classification benchmarks indicate performance comparable to that of multilayer perceptrons trained using backpropagation. More generally, these results suggest that the dynamical accessibility of fixed points and the stabilization of internal network dynamics constitute viable alternative principles for learning in recurrent systems, with conceptual links to statistical physics and potential implications for biologically motivated and neuromorphic computing architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。