用物理模型训练模拟存内计算电路,利用硬件缺陷提升能效。
Harnessing Nonidealities in Analog In-Memory Computing Circuits: A Physical Modeling Approach for Neuromorphic Systems
- 构建基于微分方程的物理神经网络,直接建模电路非理想特性。
- 在CIFAR-10上通过非理想性提升学习性能,加速20倍、内存降100倍。
- 适用于边缘智能中的低功耗深度学习,适合芯片级部署场景。
大规模深度学习模型因能耗过高而面临扩展瓶颈,限制了其在边缘智能中的应用。存内计算(IMC)通过缓解冯·诺依曼瓶颈,显著降低能耗。然而,模拟IMC固有的硬件非理想性会削弱模型性能与可靠性。本文提出一种新方法,直接训练基于常微分方程(ODE)的物理神经网络(PNN),以建模真实电路行为。为支持大规模网络训练,我们引入可微分脉冲时间离散化(DSTD)技术,使基于ODE的PNN在速度上提升20倍、内存占用降低100倍。实验表明,该方法在CIFAR-10数据集上通过利用硬件非理想性提升了学习性能。通过sky130工艺的后版图SPICE仿真验证,所提方法使模型行为与电路动态之间的差异至少降低一个数量级。该工作为利用非理想物理器件(如非易失性阻变存储器)实现高效能深度学习铺平了道路。
原文摘要 · Abstract (English)
Large-scale deep learning models are increasingly constrained by their immense energy consumption, limiting their scalability and applicability for edge intelligence. In-memory computing (IMC) offers a promising solution by addressing the von Neumann bottleneck inherent in traditional deep learning accelerators, significantly reducing energy consumption. However, the analog nature of IMC introduces hardware nonidealities that degrade model performance and reliability. This paper presents a novel approach to directly train physical models of IMC, formulated as ordinary-differential-equation (ODE)-based physical neural networks (PNNs). To enable the training of large-scale networks, we propose a technique called differentiable spike-time discretization (DSTD), which reduces the computational cost of ODE-based PNNs by up to 20 times in speed and 100 times in memory. We demonstrate that such large-scale networks enhance the learning performance by exploiting hardware nonidealities on the CIFAR-10 dataset. The proposed bottom-up methodology is validated through the post-layout SPICE simulations on the IMC circuit with nonideal characteristics using the sky130 process. The proposed PNN approach reduces the discrepancy between the model behavior and circuit dynamics by at least an order of magnitude. This work paves the way for leveraging nonideal physical devices, such as non-volatile resistive memories, for energy-efficient deep learning applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。