提出统一步长融合架构，提升边缘设备上CNN的推理效率
USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks
- 用低延迟位串行计算实现多卷积层融合
- 通过跳过无效卷积减少40%以上功耗，精度无损
- 适合对延迟敏感或资源受限的边缘场景
卷积神经网络（CNN）在诸多应用中至关重要，但在资源受限的边缘设备上部署面临挑战。本文提出用于卷积的乘积和（SOP）单元，采用低延迟从左到右的位串行算术，以最小化响应时间并提升整体性能。研究提出一种融合多个卷积层的方法，减少片外内存通信，提高整体性能。一种有效机制在ReLU层后检测并跳过低效卷积，降低功耗而不损失精度。此外，高效的区块移动策略确保融合金字塔的均匀访问。分析表明，统一步长策略提升了操作强度。两种设计满足不同需求：一种聚焦于关键任务应用的最低响应时间，另一种针对资源受限设备，具有可比延迟。该方法显著减少了冗余计算，提升了CNN在边缘设备上的部署效率。
原文摘要 · Abstract (English)
Convolutional Neural Networks (CNNs) are crucial in various applications, but their deployment on resource-constrained edge devices poses challenges. This study presents the Sum-of-Products (SOP) units for convolution, which utilize low-latency left-to-right bit-serial arithmetic to minimize response time and enhance overall performance. The study proposes a methodology for fusing multiple convolution layers to reduce off-chip memory communication and increase overall performance. An effective mechanism detects and skips inefficient convolutions after ReLU layers, minimizing power consumption without compromising accuracy. Furthermore, efficient tile movement guarantees uniform access to the fusion pyramid. An analysis demonstrates the utile stride strategy improves operational intensity. Two designs cater to varied demands: one focuses on minimal response time for mission-critical applications, and another focuses on resource-constrained devices with comparable latency. This approach notably reduced redundant computations, improving the efficiency of CNN deployment on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。