让平衡传播支持更深网络，精度逼近反向传播。
Scaling Equilibrium Propagation to Deeper Neural Network Architectures
- 用霍普菲尔德网络加残差连接扩展平衡传播
- 在CIFAR-10上达93.92%准确率，比之前高3.5%
- 适合想在类脑硬件上训练深网络的研究者
平衡传播是一种生物可解释的反向传播替代方法，其梯度计算局部化且利用收敛的RNN达到平衡状态，适合在类脑硬件上实现。但此前研究仅限于仅含全连接层或少量卷积层的小型网络，其精度显著低于同规模反向传播训练的前馈网络。本文提出霍普菲尔德-残差网络（Hopfield-Resnet），在霍普菲尔德网络中引入残差连接，并使用截断ReLU作为激活函数。该架构改进使网络层数接近此前工作的两倍。例如，Hopfield-Resnet13在CIFAR-10上达到93.92%准确率，比之前最佳结果高出约3.5%，与使用反向传播训练的Resnet13性能相当。
原文摘要 · Abstract (English)
Equilibrium propagation has been proposed as a biologically plausible alternative to the backpropagation algorithm. The local nature of gradient computations, combined with the use of convergent RNNs to reach equilibrium states, make this approach well-suited for implementation on neuromorphic hardware. However, previous studies on equilibrium propagation have been restricted to networks containing only dense layers or relatively small architectures with a few convolutional layers followed by a final dense layer. These networks have a significant gap in accuracy compared to similarly sized feedforward networks trained with backpropagation. In this work, we introduce the Hopfield-Resnet architecture, which incorporates residual (or skip) connections in Hopfield networks with clipped $\mathrm{ReLU}$ as the activation function. The proposed architectural enhancements enable the training of networks with nearly twice the number of layers reported in prior works. For example, Hopfield-Resnet13 achieves 93.92\% accuracy on CIFAR-10, which is $\approx$3.5\% higher than the previous best result and comparable to that provided by Resnet13 trained using backpropagation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。