提出新型脑启发循环网络，让等价传播学习更稳定高效
Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections
- 引入反馈调节与残差连接，降低谱半径加速收敛
- 计算成本和训练时间降几个数量级，性能媲美反向传播
- 适合构建大规模脑启发神经网络，支持物理硬件实现
类脑智能系统需要类脑学习方法。等价传播（EP）是一种具有生物合理性、在脑启发计算硬件中潜力巨大的学习框架，但现有实现存在不稳定性与极高的计算开销。受大脑结构与动态的启发，我们提出一种生物合理性的反馈调节残差循环神经网络（FRE-RNN），并研究其在EP框架下的学习性能。反馈调节通过减小谱半径实现快速收敛，显著降低EP的计算成本与训练时间，达到与反向传播（BP）相当的基准任务表现。同时，具有脑启发拓扑的残差连接有效缓解了深层RNN中反馈路径弱时出现的梯度消失问题。该方法大幅提升了EP在大规模网络中的可应用性与实用性，所开发的技术也为物理神经网络的原位学习提供指导。
原文摘要 · Abstract (English)
Brain-like intelligent systems need brain-like learning methods. Equilibrium Propagation (EP) is a biologically plausible learning framework with strong potential for brain-inspired computing hardware. However, existing im-plementations of EP suffer from instability and prohibi-tively high computational costs. Inspired by the structure and dynamics of the brain, we propose a biologically plau-sible Feedback-regulated REsidual recurrent neural network (FRE-RNN) and study its learning performance in EP framework. Feedback regulation enables rapid convergence by reducing the spectral radius. The improvement in con-vergence property reduces the computational cost and train-ing time of EP by orders of magnitude, delivering perfor-mance on par with backpropagation (BP) in benchmark tasks. Meanwhile, residual connections with brain-inspired topologies help alleviate the vanishing gradient problem that arises when feedback pathways are weak in deep RNNs. Our approach substantially enhances the applicabil-ity and practicality of EP in large-scale networks that un-derpin artificial intelligence. The techniques developed here also offer guidance to implementing in-situ learning in physical neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。