提出首个无需数据的视觉Mamba模型量化方法，解决动态激活与长程依赖难题。
OuroMamba: A Data-Free Quantization Framework for Vision Mamba
- 通过对比学习生成语义丰富的合成数据，突破Mamba的时序状态限制。
- 动态检测每步激活异常通道，实现混合精度量化，精度领先现有方法。
- 适合部署资源受限场景，尤其适用于无训练数据的模型压缩任务。
我们提出OuroMamba，首个面向视觉Mamba模型（VMMs）的数据免费后训练量化（DFQ）方法。针对VMM在量化中面临的两大挑战：（1）递归状态转移限制长程交互，导致合成数据语义薄弱；（2）激活值随时间步呈现动态异常波动，使传统静态量化失效。为此，OuroMamba设计两阶段框架：（1）OuroMamba-Gen，基于潜空间中邻域交互的块级特征，通过对比学习生成高质量合成数据；（2）OuroMamba-Quant，引入轻量级动态异常检测机制，在推理中实现混合精度量化。具体采用阈值法每步更新通道选择策略以识别异常激活。大量实验表明，该方法在视觉与生成任务中超越现有数据驱动的PTQ技术，在多种量化设置下达到最新水平。此外，通过高效GPU内核实现最高2.36倍的实际延迟加速。代码与合成数据集已公开：https://github.com/georgia-tech-synergy-lab/ICCV-OuroMamba。
原文摘要 · Abstract (English)
We present OuroMamba, the first data-free post-training quantization (DFQ) method for vision Mamba-based models (VMMs). We identify two key challenges in enabling DFQ for VMMs, (1) VMM's recurrent state transitions restricts capturing of long-range interactions and leads to semantically weak synthetic data, (2) VMM activations exhibit dynamic outlier variations across time-steps, rendering existing static PTQ techniques ineffective. To address these challenges, OuroMamba presents a two-stage framework: (1) OuroMamba-Gen to generate semantically rich and meaningful synthetic data. It applies contrastive learning on patch level VMM features generated through neighborhood interactions in the latent state space, (2) OuroMamba-Quant to employ mixed-precision quantization with lightweight dynamic outlier detection during inference. In specific, we present a thresholding based outlier channel selection strategy for activations that gets updated every time-step. Extensive experiments across vision and generative tasks show that our data-free OuroMamba surpasses existing data-driven PTQ techniques, achieving state-of-the-art performance across diverse quantization settings. Additionally, we implement efficient GPU kernels to achieve practical latency speedup of up to 2.36x. Code and synthetic dataset are available here: https://github.com/georgia-tech-synergy-lab/ICCV-OuroMamba
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。