研究神经网络在存在硬件不确定性时如何通过平衡传播学习。
Harnessing uncertainty when learning through Equilibrium Propagation in neural networks
- 利用局部神经活动实现无需数据重排的训练
- 在有限不确定性下成功训练深层网络，性能随接近临界值提升
- 适合构建自学习类脑硬件系统的研究者参考
平衡传播（EP）是一种基于局部神经活动的监督学习算法，与需要大量数据重排的反向传播不同，避免了数据移动，因而特别适合能效敏感的类脑系统。本文评估了在含物理不确定性的硬件上使用EP进行训练的能力。结果表明，深度多层神经网络可在有限不确定性下成功训练，其临界上限与训练数据集无关，并可通过中心极限定理采样来扩展。在MNIST、KMNIST和FashionMNIST数据集上，适度不确定性反而提升模型收敛速度与性能，最优表现出现在接近临界值时。该研究为在真实硬件中就地构建自学习系统提供了支持。
原文摘要 · Abstract (English)
Equilibrium Propagation (EP) is a supervised learning algorithm that trains network parameters using local neuronal activity. This is in stark contrast to backpropagation, where updating the parameters of the network requires significant data shuffling. Avoiding data movement makes EP particularly compelling as a learning framework for energy-efficient training on neuromorphic systems. In this work, we assess the ability of EP to learn on hardware that contain physical uncertainties. This is particularly important for researchers concerned with hardware implementations of self-learning systems that utilize EP. Our results demonstrate that deep, multi-layer neural network architectures can be trained successfully using EP in the presence of finite uncertainties, up to a critical limit. This limit is independent of the training dataset, and can be scaled through sampling the network according to the central limit theorem. Additionally, we demonstrate improved model convergence and performance for finite levels of uncertainty on the MNIST, KMNIST and FashionMNIST datasets. Optimal performance is found for networks trained with uncertainties close to the critical limit. Our research supports future work to build self-learning hardware in situ with EP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。