为微控制器设计稀疏神经网络加速方案，显著提升能效
Lightweight Software Kernels and Hardware Extensions for Efficient Sparse Deep Neural Networks on Microcontrollers
- 针对1:8至1:16稀疏度优化软件核，支持多核RISC-V MCU
- 在ResNet18上实现3.21倍加速，精度损失小于1.5%
- 轻量级指令集扩展带来1.9倍额外提速，面积开销仅5%
在微控制器等边缘设备上加速剪枝后的深度神经网络（DNNs）是一项挑战，因其严格的面积与功耗限制。本文提出三项贡献：首先，针对N:M剪枝层设计了一组优化的软件核，适用于超低功耗多核RISC-V MCU，相比稠密版本在1:8和1:16稀疏度下分别快2.1倍和3.4倍；其次，实现了一种轻量级指令集架构（ISA）扩展，加速间接加载与非零索引解压操作，最多带来1.9倍速度提升，代价仅为5%面积开销；最后，扩展开源DNN编译器以使用我们的稀疏核，对完整网络进行加速，在ResNet18和Vision Transformer（ViT）上分别获得3.21倍和1.81倍加速，相比稠密基线精度损失低于1.5%。
原文摘要 · Abstract (English)
The acceleration of pruned Deep Neural Networks (DNNs) on edge devices such as Microcontrollers (MCUs) is a challenging task, given the tight area- and power-constraints of these devices. In this work, we propose a three-fold contribution to address this problem. First, we design a set of optimized software kernels for N:M pruned layers, targeting ultra-low-power, multicore RISC-V MCUs, which are up to 2.1x and 3.4x faster than their dense counterparts at 1:8 and 1:16 sparsity, respectively. Then, we implement a lightweight Instruction-Set Architecture (ISA) extension to accelerate the indirect load and non-zero indices decompression operations required by our kernels, obtaining up to 1.9x extra speedup, at the cost of a 5% area overhead. Lastly, we extend an open-source DNN compiler to utilize our sparse kernels for complete networks, showing speedups of 3.21x and 1.81x on a ResNet18 and a Vision Transformer (ViT), with less than 1.5% accuracy drop compared to a dense baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。