探索一元矩阵乘法在低精度AI加速中的能效优势
Exploration of Unary Arithmetic-Based Matrix Multiply Units for Low Precision DL Accelerators
- 用一元编码替代传统二进制实现矩阵乘法，降低硬件开销
- 在低比特（如4位）下，一元设计比传统二进制快2.1倍、省电3.8倍
- 适合边缘AI场景，尤其适用于稀疏权重的轻量级模型推理
矩阵乘法（GEMM）是深度学习的核心运算。随着深度学习向低精度演进，近年出现了基于一元编码的新型GEMM硬件设计，作为传统二进制设计的替代方案。本文聚焦于面向整数型深度学习推理的一元GEMM设计，对uGEMM、tuGEMM和tubGEMM三种最新方案进行系统评估，并与传统二进制GEMM对比。通过跨不同比特宽度和矩阵尺寸的后综合严格评估，揭示各设计的权衡关系并确定最优工作点。此外，分析了八个预训练卷积神经网络（CNNs）和LLaMA2大语言模型的权重稀疏性。结果表明，一元GEMM可有效支持未来边缘AI加速器的能效计算。
原文摘要 · Abstract (English)
General matrix multiplication (GEMM) is a fundamental operation in deep learning (DL). With DL moving increasingly toward low precision, recent works have proposed novel unary GEMM designs as an alternative to conventional binary GEMM hardware. A rigorous evaluation of recent unary and binary GEMM designs is needed to assess the potential of unary hardware for future DL compute. This paper focuses on unary GEMM designs for integer-based DL inference and performs a detailed evaluation of three latest unary design proposals, namely, uGEMM, tuGEMM and tubGEMM, by comparing them to a conventional binary GEMM. Rigorous post-synthesis evaluations beyond prior works are performed across varying bit-widths and matrix sizes to assess the designs' tradeoffs and determine optimal sweetspots. Further, we perform weight sparsity analysis across eight pretrained convolutional neural networks (CNNs) and the LLaMA2 large language model (LLM). In this work, we demonstrate how unary GEMM can be effectively used for energy-efficient compute in future edge AI accelerators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。