arXiv:2602.01037cs.CVcs.AI2026-02被引 1

针对视觉语言模型专家异构性,提出双感知量化方法提升压缩效率与性能。

VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models

  • 根据专家激活频率和模态亲和度设计分层量化策略
  • 在W3A16下比当前最优方法提升2.04%~3.09%准确率
  • 适合需高效部署多模态大模型的研究者与工程师

混合专家(MoE)视觉语言模型虽性能出色,但内存与计算开销巨大,压缩至关重要。后训练量化(PTQ)是一种无需训练的高效压缩技术。现有方法忽视两个关键异质性:视觉与语言令牌的本质差异,以及不同专家贡献不均的问题。为此,我们提出视觉专家量化(VEQ),一种兼顾跨模态差异与专家异质性的双感知量化框架。具体包含:1)模态-专家感知量化,利用专家激活频率优先优化关键专家的量化误差;2)模态亲和感知量化,结合令牌-专家亲和度与模态信息构建增强海森矩阵,指导校准过程。在多个基准上的实验表明,VEQ持续优于当前最优基线。在W3A16配置下,相比先前SOTA方法,在Kimi-VL上平均准确率提升2.04%,在Qwen3-VL上提升3.09%,展现出对多种多模态任务的优异鲁棒性。代码将开源于https://github.com/guangshuoqin/VEQ。

原文摘要 · Abstract (English)

Mixture-of-Experts(MoE) Vision-Language Models (VLMs) offer remarkable performance but incur prohibitive memory and computational costs, making compression essential. Post-Training Quantization (PTQ) is an effective training-free technique to address the massive memory and computation overhead. Existing quantization paradigms fall short as they are oblivious to two critical forms of heterogeneity: the inherent discrepancy between vision and language tokens, and the non-uniform contribution of different experts. To bridge this gap, we propose Visual Expert Quantization (VEQ), a dual-aware quantization framework designed to simultaneously accommodate cross-modal differences and heterogeneity between experts. Specifically, VEQ incorporates 1)Modality-expert-aware Quantization, which utilizes expert activation frequency to prioritize error minimization for pivotal experts, and 2)Modality-affinity-aware Quantization, which constructs an enhanced Hessian matrix by integrating token-expert affinity with modality information to guide the calibration process. Extensive experiments across diverse benchmarks verify that VEQ consistently outperforms state-of-the-art baselines. Specifically, under the W3A16 configuration, our method achieves significant average accuracy gains of 2.04\% on Kimi-VL and 3.09\% on Qwen3-VL compared to the previous SOTA quantization methods, demonstrating superior robustness across various multimodal tasks. Our code will be available at https://github.com/guangshuoqin/VEQ.

量化MoE视觉语言模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。