arXiv:2603.17809cs.CVcs.AI2026-03中稿 · CVPR被引 2

用积分梯度量化每个令牌重要性,提升视觉语言模型压缩精度。

Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients

  • 以令牌为粒度,用集成梯度评估敏感度
  • 3比特量化下平均准确率提升1.60%,误差仅1.33%
  • 适合追求高精度模型压缩的开发者

大型视觉语言模型(LVLMs)在多模态任务中表现卓越,但计算与内存开销大,制约实际部署。后训练量化是降低内存和加速推理的有效手段。然而现有方法仅在模态层面衡量令牌敏感度,难以捕捉复杂的跨令牌交互,且无法在令牌层面定量评估量化误差。由于模型内部模态边界逐渐模糊,需更细粒度校准。受机制可解释性中公理化归因启发,本文提出基于量化感知积分梯度(QIG)的细粒度量化策略,将敏感度分析从模态层级推进至令牌层级,同时反映跨模态与内模态动态。在多个LVLM上,于W4A8与W3A16设置下进行实验,结果表明该方法显著提升模型准确率,延迟增加可忽略。例如,在3比特权重量化下,LLaVA-onevision-7B平均准确率提升1.60%,与全精度模型差距缩至1.33%。代码已开源:https://github.com/ucas-xiang/QIG。

原文摘要 · Abstract (English)

Large Vision Language Models (LVLMs) have achieved remarkable success in a range of downstream tasks that require multimodal interaction, but their capabilities come with substantial computational and memory overhead, which hinders practical deployment. Among numerous acceleration techniques, post-training quantization is a popular and effective strategy for reducing memory cost and accelerating inference. However, existing LVLM quantization methods typically measure token sensitivity at the modality level, which fails to capture the complex cross-token interactions and falls short in quantitatively measuring the quantization error at the token level. As tokens interact within the model, the distinction between modalities gradually diminishes, suggesting the need for fine-grained calibration. Inspired by axiomatic attribution in mechanistic interpretability, we introduce a fine-grained quantization strategy on Quantization-aware Integrated Gradients (QIG), which leverages integrated gradients to quantitatively evaluate token sensitivity and push the granularity from modality level to token level, reflecting both inter-modality and intra-modality dynamics. Extensive experiments on multiple LVLMs under both W4A8 and W3A16 settings show that our method improves accuracy across models and benchmarks with negligible latency overhead. For example, under 3-bit weight-only quantization, our method improves the average accuracy of LLaVA-onevision-7B by 1.60%, reducing the gap to its full-precision counterpart to only 1.33%. The code is available at https://github.com/ucas-xiang/QIG.

量化视觉语言模型细粒度积分梯度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。