通过优化乘法累加器的位权重维度，提升张量处理器的能效与面积效率。
Exploring the Performance Improvement of Tensor Processing Engines through Transformation in the Bit-weight Dimension of MACs
- 从乘法累加器的位权重维度切入，重构张量处理器微架构设计范式。
- 在四种经典架构上实现1.27x~1.56x面积改进和1.04x~1.56x能效提升。
- 特别适合高能效需求的边缘AI芯片与专用加速器设计者参考。
通用矩阵-矩阵乘法（GEMM）是人工智能计算的核心，使得张量处理引擎（TPEs）在GPU及专用架构中愈发关键。现有架构主要优化数据流或操作数复用策略，但考虑矩阵乘法与乘法累加器（MACs）之间的交互，具有更大优化潜力。本文提出一种新的硬件视角，聚焦于MACs的位权重维度。以矩阵三重循环为例，引入更细粒度的TPE表示法，并提出新方法设计与优化处理单元（PE）微架构。基于此表示法及其变换，提出四种优化技术，在时序、面积与功耗方面均取得改善。采用SMIC-28nm工艺在RTL层面实现，评估覆盖四种经典TPE架构：列阵（systolic array）、3D-Cube、乘法累加树（multiplier-adder tree）和2D-Matrix。所提方法分别实现1.27x、1.28x、1.56x、1.44x面积效率提升，以及1.04x、1.56x、1.49x、1.20x能效增益。应用于位片架构时，相较Laconic，能量效率提升12.10倍，面积效率提升2.85倍。完整Verilog代码及详细时序、面积、功耗报告已公开于https://github.com/wqzustc/High-Performance-Tensor-Processing-Engines。
原文摘要 · Abstract (English)
General matrix-matrix multiplication (GEMM) is a cornerstone of AI computations, making tensor processing engines (TPEs) increasingly critical in GPUs and domain-specific architectures. Existing architectures primarily optimize dataflow or operand reuse strategies. However, considering the interaction between matrix multiplication and multiply-accumulators (MACs) offers greater optimization potential. This work introduces a novel hardware perspective on matrix multiplication, focusing on the bit-weight dimension of MACs. We propose a finer-grained TPE notation using matrix triple loops as an example, introducing new methods for designing and optimizing PE microarchitectures. Based on this notation and its transformations, we propose four optimization techniques that improve timing, area, and power consumption. Implementing our design in RTL using the SMIC-28nm process, we evaluate its effectiveness across four classic TPE architectures: systolic array, 3D-Cube, multiplier-adder tree, and 2D-Matrix. Our techniques achieve area efficiency improvements of 1.27x, 1.28x, 1.56x, and 1.44x, and energy efficiency gains of 1.04x, 1.56x, 1.49x, and 1.20x, respectively. Applied to a bit-slice architecture, our approach achieves a 12.10x improvement in energy efficiency and 2.85x in area efficiency compared to Laconic. Our Verilog HDL code, along with timing, area, and power reports, is available at https://github.com/wqzustc/High-Performance-Tensor-Processing-Engines
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。