优化内存计算芯片权重布局，显著提升神经网络推理效率
Pack my weights and run! Minimizing overheads for in-memory computing accelerators
- 设计新型权重打包映射算法，减少加载时间
- 实现计算资源利用率与并行性的双重提升
- 适用于低功耗边缘设备的神经网络加速
内存计算硬件加速器相比传统数字设计，在矩阵-向量乘法（MVM）上可实现超过10倍的峰值能效和性能提升，因而备受关注用于神经网络负载加速。然而，这些优势仅在最大化计算资源利用率且最小化操作数加载开销时才能实现。为此，本文提出一种新型权重映射算法，通过高效打包网络层权重至可用内存阵列中，同时实现1）最小化权重加载时间，2）最大限度利用内存计算架构的并行性。基于MLPerf Tiny基准进行多组案例研究，结果显示该方法在IMC架构上可实现10-100倍的EDP（能量延迟积）改进。
原文摘要 · Abstract (English)
In-memory computing hardware accelerators allow more than 10x improvements in peak efficiency and performance for matrix-vector multiplications (MVM) compared to conventional digital designs. For this, they have gained great interest for the acceleration of neural network workloads. Nevertheless, these potential gains are only achieved when the utilization of the computational resources is maximized and the overhead from loading operands in the memory array minimized. To this aim, this paper proposes a novel mapping algorithm for the weights in the IMC macro, based on efficient packing of the weights of network layers in the available memory. The algorithm realizes 1) minimization of weight loading times while at the same time 2) maximally exploiting the parallelism of the IMC computational fabric. A set of case studies are carried out to show achievable trade-offs for the MLPerf Tiny benchmark \cite{mlperftiny} on IMC architectures, with potential $10-100\times$ EDP improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。