为边缘设备设计超低功耗CGRA，加速Transformer的矩阵运算。
An ultra-low-power CGRA for accelerating Transformers at the edge
- 采用4×4处理单元阵列与专用内存操作块，实现高效并行计算与数据重用。
- 通过无开关网格环形互连，降低功耗和延迟，提升数据传输效率。
- 专为边缘设备能效约束优化，适合部署复杂AI模型的嵌入式场景。
Transformers 在自然语言处理、计算机视觉等领域彻底改变了深度学习。然而，其高计算需求使得在低功耗边缘设备上部署面临挑战。本文提出一种针对边缘应用能效与资源约束而设计的超低功耗粗粒度可重构数组（CGRA）架构，专门用于加速Transformer模型中的通用矩阵乘法（GEMM）操作。该架构集成4×4处理单元（PE）阵列以实现高效并行计算，并配备专用的4×2内存操作块（MOBs），优化数据加载/存储操作，降低内存带宽需求并增强数据重用。通过无开关网格环形互连网络，实现处理单元与内存操作块间的直接通信，避免集中式交换，进一步减少功耗与延迟。该异构阵列设计与高效数据流结合，有效应对Transformer的独特计算需求，为在边缘设备上部署复杂机器学习模型提供了可扩展路径。
原文摘要 · Abstract (English)
Transformers have revolutionized deep learning with applications in natural language processing, computer vision, and beyond. However, their computational demands make it challenging to deploy them on low-power edge devices. This paper introduces an ultra-low-power, Coarse-Grained Reconfigurable Array (CGRA) architecture specifically designed to accelerate General Matrix Multiplication (GEMM) operations in transformer models tailored for the energy and resource constraints of edge applications. The proposed architecture integrates a 4 x 4 array of Processing Elements (PEs) for efficient parallel computation and dedicated 4 x 2 Memory Operation Blocks (MOBs) for optimized LOAD/STORE operations, reducing memory bandwidth demands and enhancing data reuse. A switchless mesh torus interconnect network further minimizes power and latency by enabling direct communication between PEs and MOBs, eliminating the need for centralized switching. Through its heterogeneous array design and efficient dataflow, this CGRA architecture addresses the unique computational needs of transformers, offering a scalable pathway to deploy sophisticated machine learning models on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。