通过剪枝补偿异常值，实现大模型高精度低功耗量化。
MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization
- 用剪枝分配异常值额外比特，兼顾精度与硬件效率
- 相比现有方法，推理速度提升3倍，能耗降低50%
- 无需依赖异常值局部性，适用多种大模型
基础模型的量化比传统深度神经网络更具挑战性，因其存在大量大数值异常值。现有异常值感知的软硬件协同技术要么采用混合精度保留异常值高精度但降低硬件效率，要么对异常值和正常值统一量化，虽提升效率却牺牲精度。为解决这一矛盾，我们提出MicroScopiQ，一种新型协同设计方法，利用剪枝补充异常值感知量化：在保留异常值高精度的同时，剪掉部分不重要权重以分摊额外比特，从而保证高精度并优化内存与硬件效率。我们设计了一种高吞吐、低开销的加速器架构，包含多精度整数处理单元和名为ReCoN的片上网络，有效抽象支持高精度异常值的复杂性。与以往方法不同，MicroScopiQ不假设异常值具有局部性，可广泛适用于各类基础模型。在多种量化设置下的大量实验表明，MicroScopiQ实现了最先进的量化精度，同时相比现有方案推理速度最高提升3倍，能耗降低50%。代码已开源：https://github.com/georgia-tech-synergy-lab/MicroScopiQ-LLM-Quantization
原文摘要 · Abstract (English)
Quantization of foundational models (FMs) is significantly more challenging than traditional DNNs due to the emergence of large magnitude values called outliers. Existing outlier-aware algorithm-architecture co-design techniques either use mixed-precision, retaining outliers at high precision but compromise hardware efficiency, or quantize inliers and outliers at the same precision, improving hardware efficiency at the cost of accuracy. To address this mutual exclusivity, we propose MicroScopiQ, a novel co-design technique that leverages pruning to complement outlier-aware quantization. MicroScopiQ retains outliers at higher precision while pruning a certain fraction of least important weights to distribute the additional outlier bits; ensuring high accuracy, aligned memory and hardware efficiency. We design a high-throughput, low overhead accelerator architecture composed of multi-precision INT processing elements and a network-on-chip called ReCoN that efficiently abstracts the complexity of supporting high-precision outliers. Additionally, unlike prior techniques, MicroScopiQ does not assume any locality of outlier weights, enabling applicability to a broad range of FMs. Extensive experiments across diverse quantization settings demonstrate that MicroScopiQ achieves state-of-the-art quantization accuracy, while delivering up to 3x faster inference and 2x lower energy consumption compared to existing alternatives. Code is available at: https://github.com/georgia-tech-synergy-lab/MicroScopiQ-LLM-Quantization
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。