MATCHA提升多加速器边缘芯片的深度神经网络部署效率
MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs
- 通过约束规划与模式匹配，实现异构加速器的高效并行调度
- 在MLPerf Tiny上将推理延迟降低35%，加速器利用率显著提升
- 适合需要低延迟、高吞吐的边缘设备部署场景
在具备多个异构加速引擎的系统级芯片（SoC）上部署深度神经网络（DNN）极具挑战性，现有多数部署框架无法充分挖掘异构性优势。本文提出MATCHA，一个统一的DNN部署框架，能够为并行异构加速器生成高度并发的调度方案，并采用约束规划优化L3/L2内存分配与调度。通过跨硬件单元的模式匹配、分块与映射，实现并行执行和高加速器利用率。在搭载两个异构加速器的SoC上，针对MLPerf Tiny基准测试，相比当前最先进的MATCH编译器，MATCHA将加速器利用率提升35%，推理延迟显著降低。
原文摘要 · Abstract (English)
Deploying DNNs on System-on-Chips (SoC) with multiple heterogeneous acceleration engines is challenging, and the majority of deployment frameworks cannot fully exploit heterogeneity. We present MATCHA, a unified DNN deployment framework that generates highly concurrent schedules for parallel, heterogeneous accelerators and uses constraint programming to optimize L3/L2 memory allocation and scheduling. Using pattern matching, tiling, and mapping across individual HW units enables parallel execution and high accelerator utilization. On the MLPerf Tiny benchmark, using a SoC with two heterogeneous accelerators, MATCHA improves accelerator utilization and reduces inference latency by up to 35% with respect to the the state-of-the-art MATCH compiler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。