提出针对视觉Mamba的高效量化方法，精度损失仅0.8%~1.2%
Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterization
- 采用分块缩放的逐令牌量化，提升线性与卷积层精度
- 通过重参数化简化状态空间层量化，减少误差传播
- 融合计算操作降低开销，适合部署在资源受限设备
Mamba模型基于结构化状态空间模型（SSM），具备线性时间复杂度，展现出巨大潜力。视觉Mamba（ViM）通过引入双向SSM和图像块嵌入，将该框架拓展至视觉任务，性能超越基于Transformer的模型。尽管模型量化对高效计算至关重要，现有研究仅关注原始Mamba模型，未涉及ViM的量化，且忽略了核心的SSM层量化，其固有结构导致朴素量化易引发显著误差传播。本文聚焦于ViM的后训练量化（PTQ），提出三项核心技术：1）针对线性与卷积层的k-缩放逐令牌量化；2）用于简化隐藏状态量化的重参数化技术；3）通过操作整合实现因子决定的计算开销降低。实验结果表明，在ImageNet-1k数据集上，量化仅导致0.8%~1.2%准确率下降，验证了该方法的有效性。
原文摘要 · Abstract (English)
The Mamba model, utilizing a structured state-space model (SSM), offers linear time complexity and demonstrates significant potential. Vision Mamba (ViM) extends this framework to vision tasks by incorporating a bidirectional SSM and patch embedding, surpassing Transformer-based models in performance. While model quantization is essential for efficient computing, existing works have focused solely on the original Mamba model and have not been applied to ViM. Additionally, they neglect quantizing the SSM layer, which is central to Mamba and can lead to substantial error propagation by naive quantization due to its inherent structure. In this paper, we focus on the post-training quantization (PTQ) of ViM. We address the issues with three core techniques: 1) a k-scaled token-wise quantization method for linear and convolutional layers, 2) a reparameterization technique to simplify hidden state quantization, and 3) a factor-determining method that reduces computational overhead by integrating operations. Through these methods, the error caused by PTQ can be mitigated. Experimental results on ImageNet-1k demonstrate only a 0.8-1.2\% accuracy degradation due to PTQ, highlighting the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。