arXiv:2505.17791cs.NEcs.AI2025-05被引 1

针对新型神经形态硬件,提出无需高精度的反向传播训练方法。

Bruno: Backpropagation Running Undersampled for Novel device Optimization

  • 从物理器件模型出发,构建适配铁电电容与阻变存储器的训练算法
  • 在低精度和随机性条件下仍能稳定训练,减少时间和内存开销
  • 适合研究神经形态芯片优化的工程师和硬件敏感型机器学习研究者

为提升神经形态与机器学习系统的效率,近年来聚焦于为神经网络设计专用硬件。这些系统超越通用硬件(如GPU)的冯·诺依曼架构,具备潜在性能优势。然而,针对专用硬件开发的神经网络需考虑其特定特性,这要求新的训练算法与精确的硬件模型,不能抽象为通用计算平台。本文提出自下而上的方法,基于铁电电容(FeCAPs)和阻变存储器(RRAMs)构建的紧凑器件模型,实现对硬件基脉冲神经元与突触的训练。我们开发了名为BRUNO的训练算法,可在存在随机性或低比特精度等硬件限制时仍可靠训练网络。通过与时间反向传播(Backpropagation Through Time)对比,在音乐预测(使用铁电漏电积分发放神经元,FeLIF)与手写盲文分类(使用量化RRAM突触与FeLIF神经元)任务上测试。结果表明,采用BRUNO可显著降低检测时空模式所需的时间与内存,验证其在硬件受限场景下的优势。

原文摘要 · Abstract (English)

Recent efforts to improve the efficiency of neuromorphic and machine learning systems have centred on developing of specialised hardware for neural networks. These systems typically feature architectures that go beyond the von Neumann model employed in general-purpose hardware such as GPUs, offering potential efficiency and performance gains. However, neural networks developed for specialised hardware must consider its specific characteristics. This requires novel training algorithms and accurate hardware models, since they cannot be abstracted as a general-purpose computing platform. In this work, we present a bottom-up approach to training neural networks for hardware-based spiking neurons and synapses, built using ferroelectric capacitors (FeCAPs) and resistive random-access memories (RRAMs), respectively. Unlike the common approach of designing hardware to fit abstract neuron or synapse models, we start with compact models of the physical device to model the computational primitives. Based on these models, we have developed a training algorithm (BRUNO) that can reliably train the networks, even when applying hardware limitations, such as stochasticity or low bit precision. We analyse and compare BRUNO with Backpropagation Through Time. We test it on different spatio-temporal datasets. First on a music prediction dataset, where a network composed of ferroelectric leaky integrate-and-fire (FeLIF) neurons is used to predict at each time step the next musical note that should be played. The second dataset consists on the classification of the Braille letters using a network composed of quantised RRAM synapses and FeLIF neurons. The performance of this network is then compared with that of networks composed of LIF neurons. Experimental results show the potential advantages of using BRUNO by reducing the time and memory required to detect spatio-temporal patterns with quantised synapses.

神经形态计算反向传播低精度训练硬件加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。