量化压缩会降低视觉问答模型的准确率和可靠性，但可优化。
Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs
- 用数据无关与数据相关量化方法压缩多模态大模型
- 量化使准确率和可靠性下降，但数据感知方法影响较小
- 引入选择器机制显著提升量化后模型的可靠性，适合边缘部署
多模态大语言模型（MLLM）在需要可靠性和效率的领域日益广泛应用。然而现有模型仍存在过度自信问题，产生高确定性但错误的答案。同时，其庞大的规模限制了在边缘设备上的部署，需进行压缩。本文研究后训练量化（PTQ）对视觉问答（VQA）中准确率与可靠性的影响。评估了 Qwen2-VL-7B 与 Idefics3-8B 两个 MLLM，在多种位宽下使用无数据（HQQ）与有数据（MBQ）量化方法。为缓解量化带来的可靠性下降，我们适配了选择器置信度估计器，并测试其在不同量化水平和分布外（OOD）场景下的鲁棒性。结果表明：PTQ 同时降低准确率与可靠性；数据感知方法可减轻该影响；选择器显著缓解可靠性损失。int4 MBQ 与选择器结合实现最佳效率-可靠性平衡，在约 75% 内存需求下逼近未压缩性能。本研究首次系统揭示了多模态场景中量化与可靠性之间的关联。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLM) are increasingly deployed in domains where both reliability and efficiency are critical. However, current models remain overconfident, producing highly certain but incorrect answers. At the same time, their large size limits deployment on edge devices, necessitating compression. We study the intersection of these two challenges by analyzing how Post-Training Quantization (PTQ) compression affects both accuracy and reliability in Visual Question Answering (VQA). We evaluate two MLLMs, Qwen2-VL-7B and Idefics3-8B, quantized with data-free (HQQ) and data-aware (MBQ) methods across multiple bit widths. To counteract the reduction in reliability caused by quantization, we adapt the Selector confidence estimator for quantized multimodal settings and test its robustness across various quantization levels and out-of-distribution (OOD) scenarios. We find that PTQ degrades both accuracy and reliability. Data-aware methods soften the effect thereof. The Selector substantially mitigates the reliability impact. The combination of int4 MBQ and the Selector achieves the best efficiency-reliability trade-off, closing in on uncompressed performance at approx. 75% less memory demand. Overall, we present the first systematic study linking quantization and reliability in multimodal settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。