MATCH让TVM能快速适配不同边缘设备,性能媲美定制编译器。
MATCH: Model-Aware TVM-based Compilation for Heterogeneous Edge Devices
- 用可配置的硬件抽象模型提升TVM在异构设备上的编译能力
- 在DIANA上比原版TVM快60.88倍,比专用工具HTVM快16.94%
- 适合需要快速移植DNN到多款MCU的开发者使用
在集成指令处理器与硬件加速器的异构MCU上部署深度神经网络(DNN)是TinyML领域的重要挑战。现有高性能编译工具链通常针对单一MCU家族深度定制,迁移到新平台需重写几乎全部代码;而可重定向工具链如TVM无法充分利用定制加速器,生成通用但低效代码。为此,我们提出MATCH,一种基于TVM的DNN部署框架,通过可定制的模型化硬件抽象,实现跨多种MCU处理器与加速器的敏捷移植。实验表明,仅需定义抽象硬件模型和SoC特定API,该框架在两个先进异构MCU——GAP9和DIANA上,性能即可媲美甚至超越定制工具链。在MLPerf Tiny的四个模型上,相较于原版TVM,MATCH在DIANA上将推理延迟降低高达60.88倍;相比专用于DIANA的HTVM,仍降低16.94%。在GAP9上,相比专用编译器DORY,延迟降低2.15倍,得益于对片上加速器与八核集群的协同利用。
原文摘要 · Abstract (English)
Streamlining the deployment of Deep Neural Networks (DNNs) on heterogeneous edge platforms, coupling within the same micro-controller unit (MCU) instruction processors and hardware accelerators for tensor computations, is becoming one of the crucial challenges of the TinyML field. The best-performing DNN compilation toolchains are usually deeply customized for a single MCU family, and porting to a different heterogeneous MCU family implies labor-intensive re-development of almost the entire compiler. On the opposite side, retargetable toolchains, such as TVM, fail to exploit the capabilities of custom accelerators, resulting in the generation of general but unoptimized code. To overcome this duality, we introduce MATCH, a novel TVM-based DNN deployment framework designed for easy agile retargeting across different MCU processors and accelerators, thanks to a customizable model-based hardware abstraction. We show that a general and retargetable mapping framework enhanced with hardware cost models can compete with and even outperform custom toolchains on diverse targets while only needing the definition of an abstract hardware model and a SoC-specific API. We tested MATCH on two state-of-the-art heterogeneous MCUs, GAP9 and DIANA. On the four DNN models of the MLPerf Tiny suite MATCH reduces inference latency by up to 60.88 times on DIANA, compared to using the plain TVM, thanks to the exploitation of the on-board HW accelerator. Compared to HTVM, a fully customized toolchain for DIANA, we still reduce the latency by 16.94%. On GAP9, using the same benchmarks, we improve the latency by 2.15 times compared to the dedicated DORY compiler, thanks to our heterogeneous DNN mapping approach that synergically exploits the DNN accelerator and the eight-cores cluster available on board.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。