arXiv:2505.15622cs.LG2025-05中稿 · International Join…被引 19

提出新型微型AI性能评估方法,兼顾能耗与延迟。

Benchmarking Energy and Latency in TinyML: A Novel Method for Resource-Constrained AI

  • 分预处理、推理、后处理三阶段测量能耗与延迟
  • 1000次测试验证结果,发现降电压频率提升效率
  • 适合嵌入式AI开发与跨平台选型参考

物联网兴起推动了边缘端机器学习需求,TinyML成为资源受限设备(如MCU)的可行方案。然而,因架构和应用场景多样,性能评估仍具挑战。现有方法存在诸多局限。本文提出一种新基准测试方法,整合能耗与延迟测量,并区分预推理、推理、后推理三个执行阶段。系统设计确保设备无需外部供电即可运行,支持自动化测试以提升统计显著性。以集成NPU的STM32N6 MCU为测试平台,对比高性能与低功耗两种配置。分别分析各阶段能量延迟积(EDP)变化,揭示硬件配置对能效的影响。每种模型均测试1000次,确保结果可靠。结果表明,降低核心电压与时钟频率可显著提升预处理与后处理阶段能效,且不影响网络执行性能。该方法适用于跨平台比较,可量化不同硬件实现中预/后处理开销差异。

原文摘要 · Abstract (English)

The rise of IoT has increased the need for on-edge machine learning, with TinyML emerging as a promising solution for resource-constrained devices such as MCU. However, evaluating their performance remains challenging due to diverse architectures and application scenarios. Current solutions have many non-negligible limitations. This work introduces an alternative benchmarking methodology that integrates energy and latency measurements while distinguishing three execution phases pre-inference, inference, and post-inference. Additionally, the setup ensures that the device operates without being powered by an external measurement unit, while automated testing can be leveraged to enhance statistical significance. To evaluate our setup, we tested the STM32N6 MCU, which includes a NPU for executing neural networks. Two configurations were considered: high-performance and Low-power. The variation of the EDP was analyzed separately for each phase, providing insights into the impact of hardware configurations on energy efficiency. Each model was tested 1000 times to ensure statistically relevant results. Our findings demonstrate that reducing the core voltage and clock frequency improve the efficiency of pre- and post-processing without significantly affecting network execution performance. This approach can also be used for cross-platform comparisons to determine the most efficient inference platform and to quantify how pre- and post-processing overhead varies across different hardware implementations.

TinyML能耗评估边缘计算嵌入式AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。