通过分层异构量化,让脉冲视觉变压器在低功耗设备上高效运行。
Trimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search
- 按层选用均匀或2的幂次量化,混合位数实现压缩。
- 平均有效精度3.14-3.67比特,准确率损失小于1%。
- 模型压缩8.7倍以上,能效提升超5倍,适合边缘部署。
脉冲神经网络(SNNs)因其低功耗特性,适合在边缘设备和类脑硬件上部署。近期基于SNN的视觉变压器受到关注,融合了类似人工神经网络的注意力机制,并表现出优异性能。然而,将大型未压缩高精度的脉冲变压器部署在手机等资源受限的边缘设备上仍面临巨大挑战,主要源于其高计算需求。本文提出一种新型分层异构量化方法,对脉冲变压器进行压缩。该方法为每一层优化选择统一量化或2的幂次量化两种方案,支持混合位数。实验表明,该方法可在保持高性能的同时,实现平均有效分辨率3.14-3.67比特,且在DVS Gesture和CIFAR10-DVS数据集上准确率下降不足1%。标准浮点脉冲变压器模型压缩率达8.71×至10.19×。此外,在N-Caltech101、DVS-Gesture和CIFAR10-DVS数据集上,分别实现5.69×、8.72×和10.2×的能效提升,同时保持85.3%、97.57%和80.4%的高准确率。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating attention mechanisms akin to their counterparts in Artificial Neural Networks (ANNs) while demonstrating excellent performance. However, deploying large spiking transformer models on resource-constrained edge devices such as mobile phones, still poses significant challenges resulted from the high computational demands of large uncompressed high-precision models. In this work, we introduce a novel heterogeneous quantization method for compressing spiking transformers through layer-wise quantization. Our approach optimizes the quantization of each layer using one of two distinct quantization schemes, i.e., uniform or power-of-two quantification, with mixed bit resolutions. Our heterogeneous quantization demonstrates the feasibility of maintaining high performance for spiking transformers while utilizing an average effective resolution of 3.14-3.67 bits with less than a 1% accuracy drop on DVS Gesture and CIFAR10-DVS datasets. It attains a model compression rate of 8.71x-10.19x for standard floating-point spiking transformers. Moreover, the proposed approach achieves a significant energy reduction of 5.69x, 8.72x, and 10.2x while maintaining high accuracy levels of 85.3%, 97.57%, and 80.4% on N-Caltech101, DVS-Gesture, and CIFAR10-DVS datasets, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。