用图解释技术优化神经网络层融合,减少显存访问超20%。
Applying Graph Explanation to Operator Fusion
- 基于图解释识别导致融合无效的关键操作
- 通过贪心树算法递归拆分融合组,降低显存访问
- 适用于ResNet、MobileNet等模型,显著提升推理效率
层融合技术对提升深度神经网络(DNN)推理效率至关重要,可降低加速器片上缓存与主存(DRAM)之间的数据传输开销。通过将卷积、激活等多操作合并为单个执行单元(融合组),实现高效计算。然而,片上缓存容量限制了融合组大小,全网优化需将网络划分为多个融合组。寻找最优融合组是一个复杂问题,无效解的存在阻碍传统搜索算法,需更鲁棒的策略。本文引入可解释AI中的图解释技术(GET),针对无效融合组,识别导致其无效的关键操作,并利用该信息通过贪心树算法递归拆分原始融合组,以最小化DRAM访问。该方案与常见算法结合,应用于两种层融合类型:线性缓冲深度优先(LBDF)和分支需求缩减(BRR)。实验表明,在ResNet、MobileNet等经典卷积网络上有效,尤其在EfficientNet-B3上实现超过20%的DRAM访问减少。
原文摘要 · Abstract (English)
Layer fusion techniques are critical to improving the inference efficiency of deep neural networks (DNN) for deployment. Fusion aims to lower inference costs by reducing data transactions between an accelerator's on-chip buffer and DRAM. This is accomplished by grouped execution of multiple operations like convolution and activations together into single execution units - fusion groups. However, on-chip buffer capacity limits fusion group size and optimizing fusion on whole DNNs requires partitioning into multiple fusion groups. Finding the optimal groups is a complex problem where the presence of invalid solutions hampers traditional search algorithms and demands robust approaches. In this paper we incorporate Explainable AI, specifically Graph Explanation Techniques (GET), into layer fusion. Given an invalid fusion group, we identify the operations most responsible for group invalidity, then use this knowledge to recursively split the original fusion group via a greedy tree-based algorithm to minimize DRAM access. We pair our scheme with common algorithms and optimize DNNs on two types of layer fusion: Line-Buffer Depth First (LBDF) and Branch Requirement Reduction (BRR). Experiments demonstrate the efficacy of our scheme on several popular and classical convolutional neural networks like ResNets and MobileNets. Our scheme achieves over 20% DRAM Access reduction on EfficientNet-B3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。