arXiv:2602.24059cs.CVcs.AI2026-02中稿 · CVPR被引 2

针对视觉语言模型量化中的误差补偿问题,提出按令牌自适应的专家混合方法。

Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization

  • 按令牌区分重要通道,分组使用共享与路由专家补偿误差
  • 在2B到70B参数模型上保持接近全精度的性能
  • 适合需要高效压缩大模型且关注细节还原的场景

后训练量化(PTQ)通过不重新训练完整模型即可压缩权重和激活,有效缓解视觉语言模型(VLMs)的计算与内存开销。现有方法多依赖静态识别和全局补偿敏感或异常通道,但常忽略这些通道在不同输入下的分布差异,导致量化效果不佳。本文观察到重要通道的分布和出现频率在跨模态及不同令牌间存在显著差异。为此,提出 extbf{Quant Experts (QE)}:一种面向视觉语言模型量化的令牌感知自适应误差重构方法。QE将重要通道分为令牌无关与令牌相关两类:前者采用共享专家结合低秩适配器,统一补偿全局量化误差;后者设计多个路由低秩适配器,动态响应特定令牌的局部量化误差。大量实验表明,QE在2B至70B参数规模的多种量化设置下均显著提升任务准确率,性能接近全精度模型。

原文摘要 · Abstract (English)

Post-Training Quantization (PTQ) has emerged as an effective technique for alleviating the substantial computational and memory overheads of Vision-Language Models (VLMs) by compressing both weights and activations without retraining the full model. Existing PTQ methods primarily rely on static identification and global compensation of sensitive or outlier channels, yet they often overlook the distributional differences of these important channels across inputs, leading to unsatisfactory quantization. In this work, we observe that the distributions and occurrence frequencies of important channels vary significantly both across modalities and among tokens, even within the same modality. Accordingly, we propose \textbf{Quant Experts (QE)}, a token-aware adaptive error compensation with mixture-of-experts for VLMs quantization. QE divides the important channels into token-independent and token-dependent groups. For the former, a shared expert is designed for most tokens to compensate for global quantization error using a low-rank adapter. For the latter, routed experts including multiple routed low-rank adapters are elaborated to compensate for local quantization error related to specific tokens. Extensive experiments demonstrate that QE consistently enhances task accuracy across various quantization settings and model scales, ranging from 2B to 70B parameters, while maintaining performance comparable to full-precision models.

模型量化视觉语言模型专家混合自适应补偿

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。