提出首个针对视觉状态空间模型的后训练量化框架，显著提升部署效率。
QMamba: Post-Training Quantization for Vision State Space Models
- 针对状态分布特性设计长尾偏斜与时间分组量化方法
- 4比特激活下图像分类准确率比现有方法高21.0%
- 适用于资源受限设备上的视觉SSM高效部署
状态空间模型（SSMs）作为Mamaba的关键组件，因其高效的长序列建模能力，在视觉模型中日益受到关注。由于在资源受限的边缘设备上部署SSMs存在计算成本高的问题，后训练量化（PTQ）具有实现高效部署的潜力。本文提出QMamba，据我们所知是首个针对视觉SSM的PTQ框架，基于对SSM中激活分布的分析。我们发现离散参数分布呈现长尾偏斜，隐藏状态序列则表现出高度动态变化。据此，我们设计了长尾偏斜量化（LtSQ）用于离散参数，以及时间分组量化（TGQ）用于隐藏状态，有效降低了量化误差。大量实验表明，QMamba在多种模型规模和架构的视觉模型上均优于现有先进PTQ方法。特别地，在4比特激活条件下，其在ImageNet分类任务上的表现比现有方法高出21.0%。
原文摘要 · Abstract (English)
State Space Models (SSMs), as key components of Mamaba, have gained increasing attention for vision models recently, thanks to their efficient long sequence modeling capability. Given the computational cost of deploying SSMs on resource-limited edge devices, Post-Training Quantization (PTQ) is a technique with the potential for efficient deployment of SSMs. In this work, we propose QMamba, one of the first PTQ frameworks to our knowledge, designed for vision SSMs based on the analysis of the activation distributions in SSMs. We reveal that the distribution of discrete parameters exhibits long-tailed skewness and the distribution of the hidden state sequence exhibits highly dynamic variations. Correspondingly, we design Long-tailed Skewness Quantization (LtSQ) to quantize discrete parameters and Temporal Group Quantization (TGQ) to quantize hidden states, which reduces the quantization errors. Extensive experiments demonstrate that QMamba outperforms advanced PTQ methods on vision models across multiple model sizes and architectures. Notably, QMamba surpasses existing methods by 21.0% on ImageNet classification with 4-bit activations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。