让FPGA上卷积神经网络推理持续高效，避免硬件闲置
Continuous-Flow Data-Rate-Aware CNN Inference on FPGA
- 通过动态调整数据流，让低数据率层也能持续利用硬件
- 实现接近100%的硬件利用率，比传统方法节省大量计算资源
- 适合在单个FPGA上部署复杂CNN模型如MobileNet
在深度学习推理硬件加速器中，数据流架构具备低延迟和高吞吐量的优势。这类架构将每个神经元映射到专用硬件单元，特别适合现场可编程门阵列（FPGA）实现。以往的展开式实现多聚焦于全连接网络，因其结构简单；但众所周知，卷积神经网络（CNN）在相同准确率下计算量更少。观察CNN中的数据流发现，池化层及步长大于1的卷积层会减少输出数据量，导致全并行实现中数据速率下降，使硬件单元严重闲置。本文分析了CNN的数据流特性，提出一种新型数据速率感知的连续流CNN架构设计方法。该方法通过交错低数据率信号、共享硬件单元以及合理并行化，使硬件利用率接近100%，同时达到全并行实现的吞吐量。实验表明，可显著减少算术逻辑资源，使得像MobileNet这样的复杂CNN可在单一FPGA上实现高吞吐推理。
原文摘要 · Abstract (English)
Among hardware accelerators for deep-learning inference, data flow implementations offer low latency and high throughput capabilities. In these architectures, each neuron is mapped to a dedicated hardware unit, making them well-suited for field-programmable gate array (FPGA) implementation. Previous unrolled implementations mostly focus on fully connected networks because of their simplicity, although it is well known that convolutional neural networks (CNNs) require fewer computations for the same accuracy. When observing the data flow in CNNs, pooling layers and convolutional layers with a stride larger than one, the number of data at their output is reduced with respect to their input. This data reduction strongly affects the data rate in a fully parallel implementation, making hardware units heavily underutilized unless it is handled properly. This work addresses this issue by analyzing the data flow of CNNs and presents a novel approach to designing data-rate-aware, continuous-flow CNN architectures. The proposed approach ensures a high hardware utilization close to 100% by interleaving low data rate signals and sharing hardware units, as well as using the right parallelization to achieve the throughput of a fully parallel implementation. The results show that a significant amount of the arithmetic logic can be saved, which allows implementing complex CNNs like MobileNet on a single FPGA with high throughput.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。