提出新型量化方法，显著降低深度模型推理的内存与能耗。
APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design
- 将累加部分和量化融入压缩框架，实现近无损压缩
- 在多种模型上将部分和压缩至INT8，能耗降低28%-87%
- 适用于大模型如LLaMA2-7B，适合硬件协同优化场景
深度神经网络加速器虽因模型压缩与专用数据流技术取得显著进展，但高精度部分和（PSUM）频繁访问仍带来巨大内存开销。传统压缩策略常忽略对PSUM的量化，其功耗占比可达69%。本文提出一种新型加性部分和量化（APSQ）方法，将PSUM累积过程无缝集成到量化框架中，并结合可重构架构设计分组策略。APSQ在BERT、Segformer、EfficientViT等模型的NLP与视觉任务上实现近乎无损效果，将PSUM压缩至INT8，能量消耗降低28%-87%。在LLaMA2-7B上的扩展实验验证了该方法在大语言模型中的潜力。代码已开源：https://github.com/Yonghao-Tan/APSQ。
原文摘要 · Abstract (English)
DNN accelerators, significantly advanced by model compression and specialized dataflow techniques, have marked considerable progress. However, the frequent access of high-precision partial sums (PSUMs) leads to excessive memory demands in architectures utilizing input/weight stationary dataflows. Traditional compression strategies have typically overlooked PSUM quantization, which may account for 69% of power consumption. This study introduces a novel Additive Partial Sum Quantization (APSQ) method, seamlessly integrating PSUM accumulation into the quantization framework. A grouping strategy that combines APSQ with PSUM quantization enhanced by a reconfigurable architecture is further proposed. The APSQ performs nearly lossless on NLP and CV tasks across BERT, Segformer, and EfficientViT models while compressing PSUMs to INT8. This leads to a notable reduction in energy costs by 28-87%. Extended experiments on LLaMA2-7B demonstrate the potential of APSQ for large language models. Code is available at https://github.com/Yonghao-Tan/APSQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。