提出可动态适配矩阵计算的新型指令集,显著提升深度学习模型性能。
A Flexible Instruction Set Architecture for Efficient GEMMs
- 设计解耦架构的矩阵指令扩展,支持多维向量化与寄存器复用。
- 在常见模型上相比最优现有方案提速1.35倍,突破固定矩阵指令瓶颈。
- 仅需少量新增指令和寄存器,易集成于现有处理器架构中。
通用矩阵乘法(GEMM)在高性能计算与深度学习中广泛存在。传统高端CPU依赖单指令多数据(SIMD)或向量指令集加速GEMM,但对小、高或细长矩阵效率低下。近年硬件厂商推出专用矩阵指令集,虽提升吞吐量,却因结构僵化难以适应不同数据格式。本文指出,现有矩阵指令集在主流卷积与Transformer模型上表现不佳。为此提出矩阵区块扩展(MTE),首个完全解耦指令集与微架构的矩阵指令集,可无缝对接现有向量指令集。MTE仅需少量新增指令与一个64位控制状态寄存器(CSR)维持状态,实现:一、在矩阵三个维度(M, N, K)上向量化;二、利用现有向量寄存器文件容量;三、将区块形状与底层微架构解耦。实验表明,MTE相较最佳现有矩阵指令集实现1.35倍加速。
原文摘要 · Abstract (English)
GEneral Matrix Multiplications (GEMMs) are recurrent in high-performance computing and deep learning workloads. Typically, high-end CPUs accelerate GEMM workloads with Single-Instruction Multiple Data (SIMD) or vector Instruction Set Architectures (ISAs). Since these ISAs face significant issues when running GEMM workloads, particularly when dealing with small, tall, or skinny matrices, matrix ISAs have been proposed and implemented by major hardware vendors in the last years. Although these matrix ISAs deliver larger throughput when running GEMMs than their SIMD/vector counterparts, they are rigid solutions unable to dynamically adapt themselves to application-specific aspects like the data format. This paper demonstrates that the state-of-the-art matrix ISAs deliver suboptimal performance when running the most commonly used convolution and transformer models. This paper proposes the Matrix Tile Extension (MTE), the first matrix ISA that completely decouples the instruction set architecture from the microarchitecture and seamlessly interacts with existing vector ISAs. MTE incurs minimal implementation overhead since it only requires a few additional instructions and a 64-bit Control Status Register (CSR) to keep its state. Specifically, MTE can i) vectorize GEMMs across the three dimensions M, N, and K; ii) leverage the capacity of the existing vector register file; and iii) decouple the tile shape from the underlying microarchitecture. MTE achieves speed-ups of 1.35x over the best state-of-the-art matrix ISA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。