PIM-LLM用混合架构加速1比特大模型，性能提升80倍。
PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs
- 结合模拟存内计算与数字阵列，分担低精度和高精度矩阵乘运算
- 相比传统加速器，每秒生成字节数提升80倍，能效提高70%
- 适合追求极致推理速度与能效的1比特大模型部署场景
本文提出PIM-LLM，一种用于加速1比特大语言模型（LLMs）的混合架构。该架构分别利用模拟存内计算（PIM）结构加速投影层中的低精度矩阵乘法（MatMul），以及数字脉动阵列加速注意力头中的高精度矩阵乘法。实验表明，PIM-LLM相较传统硬件加速器，可实现约80倍的每秒生成字节数提升，并在能效上提高70%。此外，相较于先前基于PIM的LLM加速器，其性能在GOPS和GOPS/W上分别至少提升2倍和5倍，创下新基准。
原文摘要 · Abstract (English)
In this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。