用脉冲神经网络让大模型低能耗运行,不需反量化且保持精度。
Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation
- 将显著激活值转为脉冲计数,实现动态混合精度存储。
- 在W4A4量化下接近FP16精度,能耗降低最多4.6倍。
- 适合边缘设备部署,尤其对能效敏感的场景。
大语言模型(LLM)在边缘设备部署时依赖权值-激活量化以降低内存和计算位宽。然而,能耗受限硬件仍面临三大挑战:(1) 即使量化后,乘累加(MAC)操作仍不可避免,主导能耗;(2) 反量化(或逐张量/通道重缩放)引入额外计算与数据移动,增加延迟与能耗;(3) 统一位宽会压制显著值,而当前矩阵硬件难以支持通道内混合精度。相较之下,类脑脉冲神经网络(SNN)基于二值脉冲表示与积分发放(IF)机制,天然支持混合精度存储与节能计算,以时间累积(ACC)替代复杂MAC。受此启发,我们提出SpikeQuant:对具有显著值的激活选择性应用混合精度量化,并将其重编码为二值脉冲计数,实现不同位宽的动态混合存储。同时,通过将量化尺度嵌入IF机制阈值,实现权重与激活的节能线性变换,避免显式反量化。实验表明,SpikeQuant在W4A4量化下持续达到近FP16困惑度,相比现有方法能耗降低最多4.6倍,证明其在高精度、低能耗部署上的有效性。
原文摘要 · Abstract (English)
In the era of large language models (LLMs), weight-activation quantization helps fit models on edge device by reducing memory and compute bit-widths. However, three challenges persist for energy constrained hardware: (1) even after quantization, multiply-accumulate (MAC) operations remain unavoidable and continue to dominate energy consumption; (2) dequantization (or per-tensor/channel rescaling) introduces extra arithmetic and data movement, increasing latency and energy; (3) uniform parameters bit widths clip salient values-while intra-channel mixed precision is generally impractical on current matrix hardware and memory. In contrast, brain-inspired Spiking Neural Networks (SNNs), owing to their binary spike-based information representation and the Integrate-and-Fire (IF) paradigm, naturally support mixed-precision storage and energy-efficient computation by replacing complex MACs with temporal Accumulate (ACCs). Motivated by this property, we propose SpikeQuant, which selectively applies mixed-precision quantization to activations with salient values and re-encodes them into binary spike counts, thereby enabling dynamic mixed storage of different bitwidths. Furthermore, by embedding the quantization scale into the threshold of the IF mechanism, our approach performs energy-efficient linear transformations on weights and activations while avoiding explicit dequantization. Experimental results demonstrate that SpikeQuant consistently achieves near-FP16 perplexity under W4A4 quantization while reducing energy cost by up to 4.6 times compared to existing methods, highlighting its effectiveness for accurate and energy-efficient LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。