通过近似计算加速微控制器上的小型神经网络推理。
Accelerating TinyML Inference on Microcontrollers through Approximate Kernels
- 拆解卷积层操作数，离线评估重要性并跳过低贡献计算。
- 在STM32上实现平均21%延迟降低，准确率无损失。
- 适合资源受限的物联网设备部署，尤其对实时性要求高场景。
基于微控制器的物联网设备快速发展，广泛应用于智能制造和个性化医疗等领域。尽管微型机器学习（TinyML）中普遍采用低功耗微控制器单元（MCUs），但其性能与内存（RAM、Flash）仍面临显著限制。本文结合近似计算与软件内核设计，加速MCU上近似卷积神经网络（CNN）的推理。框架首先解包卷积层的操作数，并离线计算每个操作数的重要性；随后通过设计空间探索，基于重要性采用计算跳过策略进行近似。在STM32-Nucleo开发板及两个在CIFAR-10数据集上训练的流行CNN模型上的评估表明，相比最先进精确推理，本方法的帕累托最优解平均可降低21%延迟，且Top-1分类准确率无下降；在允许更低准确率的场景下，加速效果更显著。
原文摘要 · Abstract (English)
The rapid growth of microcontroller-based IoT devices has opened up numerous applications, from smart manufacturing to personalized healthcare. Despite the widespread adoption of energy-efficient microcontroller units (MCUs) in the Tiny Machine Learning (TinyML) domain, they still face significant limitations in terms of performance and memory (RAM, Flash). In this work, we combine approximate computing and software kernel design to accelerate the inference of approximate CNN models on MCUs. Our kernel-based approximation framework firstly unpacks the operands of each convolution layer and then conducts an offline calculation to determine the significance of each operand. Subsequently, through a design space exploration, it employs a computation skipping approximation strategy based on the calculated significance. Our evaluation on an STM32-Nucleo board and 2 popular CNNs trained on the CIFAR-10 dataset shows that, compared to state-of-the-art exact inference, our Pareto optimal solutions can feature on average 21% latency reduction with no degradation in Top-1 classification accuracy, while for lower accuracy requirements, the corresponding reduction becomes even more pronounced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。