提出新架构加速边缘设备上的反卷积运算，提升生成模型效率。
Accelerating Transposed Convolutions on FPGA-based Edge Devices
- 用矩阵乘法结合列转图像方法优化反卷积计算
- 在261种配置下平均提速1.9倍，最高达4.2倍
- 适合资源受限的生成式AI边缘部署
反卷积（TCONV）在生成式人工智能模型中实现特征图上采样。然而，主流的输入导向映射（IOM）方法存在输出映射复杂、累加重叠和无效计算等问题，加剧了资源受限边缘设备上TCONV及生成模型的性能瓶颈。为此，本文提出MM2IM——一种软硬件协同设计的加速器，结合矩阵乘法（MatMul）与col2IM，高效处理TCONV层。基于SECDA-TFLite工具链实现并评估其在261个TCONV配置下的表现，相比双线程ARM Neon优化的CPU基线平均提速1.9倍。进一步在知名生成模型的TCONV层测试中，最高提速达4.2倍，并在同类型资源受限加速器中，至少领先2x GOPs/DSP。在DCGAN和pix2pix GAN模型上评估，相较CPU基线最高提速3倍，能耗降低2.4倍。
原文摘要 · Abstract (English)
Transposed Convolutions (TCONV) enable the up-scaling mechanism within generative Artificial Intelligence (AI) models. However, the predominant Input-Oriented Mapping (IOM) method for implementing TCONV has complex output mapping, overlapping sums, and ineffectual computations. These inefficiencies further exacerbate the performance bottleneck of TCONV and generative models on resource-constrained edge devices. To address this problem, in this paper we propose MM2IM, a hardware-software co-designed accelerator that combines Matrix Multiplication (MatMul) with col2IM to process TCONV layers on resource-constrained edge devices efficiently. Using the SECDA-TFLite design toolkit, we implement MM2IM and evaluate its performance across 261 TCONV problem configurations, achieving an average speedup of 1.9x against a dual-thread ARM Neon optimized CPU baseline. We then evaluate the performance of MM2IM on a range of TCONV layers from well-known generative models achieving up to 4.2x speedup, and compare it against similar resource-constrained TCONV accelerators, outperforming them by at least 2x GOPs/DSP. Finally, we evaluate MM2IM on the DCGAN and pix2pix GAN models, achieving up to 3x speedup and 2.4x energy reduction against the CPU baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。